{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 为什么要引入核函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "处理分类问题时，由于维度增多或者数据分布在原始的特征空间Rd不是线性可分的, svm希望通过一个映射ϕ: Rd → Rd˜, 使得数据在新的空间Rd˜是线性可分的.在支持向量机的对偶问题中, 被映射到高维的特征向量总是以成对内积的形式存在,优化函数这天生的就能与核函数进行结合。即 ϕ(xi)⊤ϕ(xj).通过构造核函数转化成 k(xi,xj).因此需要引入核函数处理线性不可分问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 给出合页损失的数学形式并画出图形"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "合页损失：L(y(w⋅x+b))={1−y(w⋅x+b) if y(w*x+b)<1,\n",
    "                       0          if y(w*x+b)>=1}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "观察函数形式可以发现当y（w*x+b）大于1时（即满足约束条件）损失值为0，当其小于1时损失为1-y（w*x+b）的形式，则其函数图像类似书的合页因此名字为合页损失"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x1be59112668>]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ8AAAEoCAYAAACHLfxZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xt4VNWh/vHvmtxmGArFKI9SvPUICo0SbL1UTYhJuFkOCBVRCVAveEHyq+VRFLQKglrxUirEWlSkgNEqFgjxqCAgiKcFq3BoBakK4oWLwQCSZJII2b8/Fk4FAoQks/dc3s/z5JHJ7Nl5E5O82XuvvZZxHAcRERE3+bwOICIiiUflI9JExpgBxpgfeZ1DJJYYnXYTaTxjTBtgB/ZnSX/MiTSQflhEmmY68A1gjDFdvA4jEitUPiKNtP+opy9w4/53PedhHJGYotNuIo1kjHkFuNBxnB8ZYxzAAbo4jvNPj6OJRD0d+Yg03sfAkO89fhJIAzDGtDbGVBhjQsaYamPMWwe/2BjzA2PMZ8aY2v3bXuJObBHv6chHpBkYYxzHccz3HhugreM4240xAaAM+LXjOM9+b5sXgbMdx/mJMeYPQH/HcU5xPbyIB3TkIxIBjrV9/8MW2J+1g//Sywcm7//3aKD9/tISiXsqH5EIMcakGGNC2KHYax3HmX7QJi2B1QCO49QA+4AO7qYU8YbKRyRCHMf51nGcAHAacKYx5vIGvKwusqlEokNyE1+fsBeMtm6F7Gz46itYuhTOPdfrROK2goIC5s6d+/131fvz4DgOl156KcFg8ICN09PTefDBB98FqK6uxhjD3r17P4pcYhFXNOjUsY58Gumkk2DxYvjhD6FHD/jXv7xOJG6bPXs2lZWVVFZWHvLc+vXr2bx5MwDl5eW8//77nHvQXyi5ublMnmwv+YwePZof/ehH+Hz6kZTE0NTRbgl75POdTz6BrCyoq4O334YOOmOfkIwxfP9nac6cOQwdOhTHcXAch4suuoglS5aQnZ1NVlYWDzzwALt27SIjI4Pt27eTlpZGaWkpOTk53n0SIs2jQUc+Kp9msG4ddOsGgQAsXw6nneZ1InHbweUjksB02s0tnTvDokWwZw/k5cGXX3qdSEQkuql8mklmJrz+uh2AkJ9v/ysiIvVT+TSjCy6AV1+FzZvtIITycq8TiYhEJ5VPM8vOhnnzYP166N0bvvnG60QiItFH5RMBPXrAyy/D++9Dnz5Qz0hcEZGEpvKJkL59YfZseOcd6N8fqqu9TiQiEj1UPhE0aBA8+6wdCXfllfDtt14nEhGJDiqfCPvVr6CoCBYsgIIC2LfP60QiIt5r6txu0gAjRkBVFdxxh70Rdfp00CwqIpLIVD4uuf12O/Bg3Dho0cIeDWnlFhFJVCofF917ry2gRx6BYBAmTVIBiUhiUvm4yBh4+GF7Cu7RR20BjRvndSoREfepfFxmDDzxhC2g8ePtKbjRo71OJSLiLpWPB3w+ePppCIXgzjttAY0c6XUqERH3qHw8kpQEM2faAiostAV03XVepxIRcYfW8/FYTY2dDWHRInj+ebj6aq8TSWNoPR+RMC0mFyuqquwkpO+8A3PmwOWXe51IjpXKRyRM5RNL9uyB7t1h9WooKYGePb1OJMdC5SMSpvKJNTt3Qm4ufPghvPYa5OR4nUgaSuUjEqbyiUVlZdCtG3z2Gbz5Jlx4odeJpCFUPiJhKp9YtXUrZGXBjh2wdCl07ep1IjkalY9ImMonlm3ebAsoFIJly6BzZ68TyZGofETCVD6x7uOPbQEBLF8OHTp4m0cOT+UjEtag8tHE/lHsjDNg8WLYuxfy8uzRkIhIPFD5RLnOnWHhQjsUOy8PtmzxOpGISNOpfGJA16526PX27ZCfb0fESXRbuXIlbdq0IS0tDb/fz4ABAw7ZZvLkyRhjCAQCBAIBcnNzPUgq4g2VT4y48EIoLYVNm6BHD3tPkESvtLQ0pk6dSk1NDRs3bqS0tJSSkpJDtmvbti2hUIhQKMSSJUs8SCriDZVPDOnWDebNg3Xr7HQ8e/Z4nUgOJzMzk8GDBwPQrl070tPTWbduncepRKKHyifG9OwJL70E//gH9Olj54WT6LZixQrKysooKCg45LmysjICgQAnnHAC8+fP9yCdiDdUPjGoXz+YNQvefhv697czY0t02rZtGz169GDUqFG0b9/+gOeuuOIKtmzZQigUYuTIkQwcONCjlCLu030+MWz6dLj+eltGL78MKSleJ0osBQUFzJ07F4CqqqpD7vOpqqrilFNO4eKLL27QUU1ycjIffPABZ555ZkTyirhEN5kmgqIiuwrqoEF2PaCkJK8TJaaDbzKtq6ujQ4cOtGrVitWrV9f7mrVr15KRkYHP5+O5555j+PDh1NbW4vPphITEtAaVj1YyjXG33gqVlf9ZjvuZZ+wy3eKtp556io0bN+L3+wkEAgDcddddbNiwAYDi4mImTJjAggULMMaQlJREUVGRikcSho584sR998H999symjIFTIP+9pDmoul1RMJ05JNIxo2zI98efdQeAT38sApIRKKXyidOGAOTJtkCeuQRCAbt0ZCISDRS+cQRY+wpt6oqeyQUDMLtt3udSkTkUCqfOOPz2UEHVVVwxx32FNyIEV6nEhE5kMonDiUlwezZUF1tByAEAnDttV6nEhH5D412i2PV1dC3r10TqLjY3gskkaHRbiJhuslU7Om3Xr3gb3+DV16xZSTNT+UjEqbyEeubb6B7d1izBhYssEsySPNS+YiEqXzkP3buhEsvhX//G15/HbKzvU4UX1Q+ImEqHznQV1/ZNYG++ALefBMuuMDrRPFD5SMSpvKRQ335pT3qKS+HpUshM9PrRPFB5SMSpvKR+n36qS2gUAiWLYPOnb1OFPtUPiJhKh85vI8+sgVkDCxfDmec4XWi2KbyEQlrUPlo/vYE1aGDve5TWwt5efDZZ14nEpFEovJJYD/5CSxcCLt32wLautXrRCKSKFQ+Ce7cc+G112zx5OdDWZnXiUQkEah8hJ//HEpLYeNG6NkTdu3yOpGIxDuVjwCQkwNz58K//gW9e8OePV4nEpF4pvKRsF694C9/gXfftXPAhUJeJxKReKXykQP07w+zZtn7fwYMgJoarxOJSDxS+cghrr4ann7azgF31VXw7bdeJxKReKPykXpdfz088QTMmwfDhsG+fV4nEpF4opVM5bAKC+16QHfdZZfjnjbNLtMtItJUKh85ojvvhMpKmDDBLsf9xBN2Sh4RkaZQ+chRjR9vC+jxxyEYhIceUgGJSNPoJIoclTHw6KNw883w8MMwcaLXiaLfypUradOmDWlpafj9fgYMGHDINnV1dXTp0oWUlBQCgQDPP/+8B0lFvKHykQYxBoqKYOhQuPdeeOwxrxNFt7S0NKZOnUpNTQ0bN26ktLSUkpKSA7aZMGECW7ZsoaamhilTpnDTTTd5lFbEfSofaTCfD559FgYOhNtvhz/+0etE0SszM5PBgwcD0K5dO9LT01m3bt0B27z44otcddVV+Hw+brjhBmpra1mzZo0XcUVcp/KRY5KcDLNnQ58+MGIE/PnPXieKfitWrKCsrIyCgoID3l9eXk6nTp3Cj1u2bMnatWvdjifiCZWPHLPUVHj5ZTsL9nXXwUsveZ0oem3bto0ePXowatQo2rdvf8Bz9S0+59NYdkkQ+k6XRvH77Q2oF10EgwfDggVeJ3JfQUEBwWCQYDBY7/NVVVVkZGTQvXt3Jk2adMjz6enprF+/Pvy4oqKCjIyMiOUViSZaRlua5Jtv7BHQ//2fXZahe3evE3nj4GW06+rq6NChA61atWL16tX1vmbcuHEUFRWxfft2pk+fzm233UZFRYVbkUUipUE3Yqh8pMnKy+HSS+Gjj+CNNyAry+tE7ju4fJ588kluvfVW/H5/+H133XUXGzZsAKC4uDg81PrDDz8kKSmJadOmMXToUNezizQzlY+456uvoFs3+PJLePNNOP98rxO56+DyEUlgKh9x15dfQnY27NwJS5dCly5eJ3KPykckTOUj7vv0U3varabGrgn0vZHEcU3lIxLWoPLRaDdpVqedBosX2xtS8/Phk0+8TiQi0UjlI82uY0d73aemBvLy4PPPvU4kItFG5SMRkZFhR77t3GkLaNs2rxOJSDRR+UjE/PSn8NprsGWLPQW3Y4fXiUQkWqh8JKIuughKSuDjj6FnT9i1y+tEIhINVD4Scbm58Ne/wj//CZddBrqJX0RUPuKKyy6DF1+EVaugb18IhbxOJCJeUvmIawYMsEswvPUW/PKXdjSciCQmlY+4avBg+NOf7ECEa66BvXu9TiQiXlD5iOuGD4fJk+11oF/9Cvbt8zqRiLgt2esAkph+/WuoqoKxYyEQgGnTwDRoUg4RiQcqH/HMmDFQWQkPPAAtWtijIRWQSGJQ+YinJkywBTR5MgSD8OCDXicSETeofMRTxsDjj9uh1w89ZAvo7ru9TiUikabyEc8ZA08+aa8B3XOPPQX3m994nUpEIknlI1HB54Pp0+0R0KhRtoBuusnrVCISKSofiRrJyfD887aAbrnFFtCQIV6nEpFI0EqmEnWqq6FPH7sU94svwsCBXic6Oq1kKhKmZbQldlVW2lmwV66EuXNtGUUzlY9ImMpHYtvu3XYdoH/+E0pL7b+jlcpHJEzlI7Hv668hJwc2brQro15yideJ6qfyEQlT+Uh82L4dsrNh61ZYvBjOO8/rRIdS+YiEqXwkfnzxhS2gXbvskgznnON1ogOpfETCGlQ+mtVaYkL79vaop0UL6N4dPvzQ60RH1rFjR3w+H36/v97nJ0+ejDGGQCBAIBAgNzfX5YQi3lL5SMw4/XRbQGAHH2za5G2eIyksLGTWrFlH3KZt27aEQiFCoRBLlixxKZlIdFD5SEw580xYtMjeiJqba0/HRaPCwkJOPfVUr2OIRC2Vj8Scc86xI9/KyyEvzw5IiEVlZWUEAgFOOOEE5s+f73UcEVepfCQm/exn8D//Y4988vPtkOxYcsUVV7BlyxZCoRAjR45kYCxM4yDSjFQ+ErMuvhhKSuCjj+xsCLt3u/vxCwoKCAaDBIPBY35t+/btOfHEEwG47777qKurY8OGDc0dUSRqqXwkpuXlwSuvwNq1cNllUFHh3seePXs2lZWVVFZWHvNr165dS11dHQDPPfccAB06dGjWfCLRTLNaS8z7xS+guBgGDYJ+/exUPIGAt5lOPfVUvvjiC+rq6khKSmLIkCHU1tYCUFxczIQJE1iwYAHGGJKSkigqKsLn09+Ckjh0k6nEjVmzYNgw6N3bTkaamurex9ZNpiJhuslUEsuQIfDUU3YgwjXXwN69XicSkcNR+UhcufFG+P3v7XWga6+F/ZdVRCTK6JqPxJ3bbrPrAd1zj52O56mnwDToRICIuEXlI3Hp7rttAT30kC2gxx9XAYlEE5WPxK0HHoCqKpg8GYJBmDjR60Qi8h2Vj8QtY+z1n6oqW0QtWsDYsV6nEhFQ+UicMwb++EdbQHffbQvottu8TiUiKh+Je0lJMGOGnQn7N7+xp+CGD/c6lUhi002mkjBqa6F/f3jtNZg5EwoKmm/fuslUJEzLaIscLBSCPn1g2TL4y1/gl79snv2qfETCVD4i9amosLNgv/suzJtnJyRtKpWPSJjKR+Rwdu+2K6F+8IGdjic3t2n7U/mIhKl8RI7k668hJwc2bYKFC+Giixq/L5WPSJjKR+Rotm2Dbt3sf5csgZ/+tHH7UfmIhKl8RBri888hKwv27IG33oKzzz72fah8RMJUPiINtXGjLaB9+2D5cujY8dher/IRCdN6PiIN9eMfw+LFdgmGvDx7HUhEIkflI7LfWWfBm2/a2bDz8uCLL7xOJBK/VD4i33POOfDGG7BjB+Tnw/btXicSiU8qH5GDnHcevPoqfPYZdO8O5eVeJxKJPyofkXpkZUFJCfz733Y2hN27vU4kEl9UPiKHkZ8Pc+bAmjXwi1/Ya0Ei0jxUPiJH0KcPFBfD3/4Gl18O1dVeJxKJDyofkaMYOBCmT7cj4QYOtEsziEjTqHxEGmDYMLsiammpXQdo716vE4nENpWPSAPdfDM89hi8/DJcd529IfVwOnbsiM/nw+/31/t8XV0dXbp0ISUlhUAgwPPPPx+h1CLRSeUjcgxGjYL774dZs+DWW+FwM+oUFhYya9asw+5nwoQJbNmyhZqaGqZMmcJNN90UocQi0Ulzu4kcI8eBMWPg4YdtGT36KPh8h87ttmLFCvLz86muZ5RCp06dyM/PZ8qUKQCkpqayatUqMjMzXfkcRCKoQXO7JUc6hUi8MQYeegiqquDxxyEYPPZ9lJeX06lTp/Djli1bsnbtWpWPJAyVj0gjGAOTJ9sCmjDh2F9f3xkHn09nwSVx6LtdpJGGDi3ghReCJCXZQ58nnmj4a9PT01m/fn34cUVFBRkZGc0dUSRqqXxEGmn27NlUVlYSCtmpD379a3j22Ya9dtCgQbz44ovU1dXxzDPPkJqaqlNuklB02k2kiVJS7H979YLhwyEQgDFjTuWLL76grq6OpKQkhgwZQu3+u1OLi4u59957eeWVV0hLSyMpKYlp06Z5+BmIuE+j3USagTGGqiqHyy6Dt9+29wL17+91KhFPaBltEbd8t4z2nj3Qowe89x7Mnw+9e3udTMR1Kh8Rt3xXPgC7dkFuLqxfD6+9Bjk53mYTcZnKR8Qt3y8fsCuh5uTAp5/CokXw8597Fk3EbSofEbccXD4AW7dCdjZ89RUsXQrnnutROBF3qXxE3FJf+YBdijsryy5E99ZboFt5JAGofETccrjyAfjkE1tAdXWwfDl07OhyOBF3Nah8dJOpSIT913/B4sW2fPLy7HUgkUSn8hFxQadOduBBRYUtoC+/9DqRiLdUPiIu6dIF3ngDysogP98ORBBJVCofERedfz68+ips3mxvRi0v9zqRiDdUPiIuy8qysx+sX2/ng/vmG68TibhP5SPige7d7fxvq1dDnz52KLZIIlH5iHikb1+YPRveecdOQlrPatsicUvlI+KhQYNg+nQ7Eu7KK+Hbb71OJOIOlY+Ix4YNg6IiWLAACgpg3z6vE4lEnhaTE4kCI0ZAVRXccYddjG76dPDpT0OJYyofkShx++22gO67D1q0sEdDpkETlYjEHpWPSBT57W/tyLdJk2wBPfKICkjik8pHJIoYA7/7nT0CeuwxCAZh/HivU4k0P5WPSJQxBv7wB3sEdP/9toBGj/Y6lUjzUvmIRCGfD55+GkIhuPNOewpu5EivU4k0H5WPSJRKSoKZM20BFRbaArruOq9TiTQPLSYn0gyOtJhcU9XUQL9+sHAhPP88XH11RD6MSHPRSqYibolk+YAdgHDZZbBiBcyZA5dfHrEPJdJUKh8Rt0S6fAD27LHLMLz/vp0Vu1eviH44kcbSMtoiXpo4cSKpqamkpKTQq56muOGGG/D5fAQCAQKBAMOGDTvi/n7wA3jtNejc2U5E+tZbEQou4gKVj0gE1NbWMn78eBYuXMjOnTtZtmwZJSUlh2yXkZFBKBQiFArx5z//+aj7/eEP7bWfH//YLsXw979HIr1I5Kl8RCJgxowZtG7dmpycHFq2bEl2djZFRUXNsu8TToA334STTrKn3lavbpbdirhK5SMSARs2bCA9PT38+PTTT2fr1q2HbPfBBx8QCARo3749K1eubPD+TzoJFi+G1q3tdaAPPmiW2CKuUfmIREBDBh/ceeed7Ny5k1AoRHZ2dr3XhY7klFNsAaWkQH4+fPRRY9OKuE/lI9JIBQUFBINBgsHgIc+dddZZfP311+HHmzZt4qSTTjpgmw4dOtCqVSvAnqbbtWvXMWc44wx7Cm7vXsjLg82bj3kXIp5Q+Yg00uzZs6msrKSysvKQ54YOHcru3btZvnw5FRUVLF++nBEjRhywzZo1a8L//u1vf1tviTVE5852EMKePbaAtmxp1G5EXKXyEYkAv9/PPffcQ35+Pm3atOGSSy6hX79+ZGdnc/fddwNwyy234Pf7CQQCTJs2jZdeeqnRH69rV3j9ddi+3Z6CKytrrs9EJDJ0k6lIM3DjJtOGWLYMeveGjh1h6VJo08brRJKAdJOpSKLp1g3mzoX1620J7dnjdSKR+ql8ROJMz57w0kvwj3/YG1GrqrxOJHIolY9IHOrXD2bPhrfftlPx1NR4nUjkQCofkTh11VXw7LN2JNyVV8K333qdSOQ/VD4icezaa2HqVCgpgSFDYN8+rxOJWFrJVCTO3Xqrve4zejQEAvZoyKc/O8VjKh+RBHDHHVBZCePH2+W4p04F06ABsSKRofIRSRD33WcL6NFHIRiEhx9WAYl3VD4iCcIYmDTJnoJ75BFbQPfd53UqSVQqH5EEYgxMmWILaNw4ewrujju8TiWJSOUjkmB8PnjmGQiF7CCEFi3soAQRN6l8RBJQUhLMmgXV1TBypC2ga6/1OpUkEk0sKtIMomVi0WNVUwN9+9o1gYqLYdAgrxNJHGjQMBaVj0gziNXyAXv9p3dv+N//hTlz7NQ8Ik2g8hFxSyyXD9jZr/PzYc0aOxtCz55eJ5IYpvIRcUuslw/Azp2QmwsbNtiF6bKzvU4kMUrlI+KWeCgfsCugdusGn39urwNdcIHXiSQGqXxE3BIv5QOwZYs96vn6a7saamam14kkxqh8RNwST+UDsHkzZGXZe4GWLYPOnb1OJDEk8stoZ2VlEQgECAQCpKen8/777zdldzHpvPPOIy0tjUAgQLt27di8ebPXkVw3atQo/H4/xhhmzpzpdRxXTZw4kdTUVAB69erlcZrmc+qpsHgxJCfbgQgff3z4bTt27IjP58Pv97sXMMqsXLmSNm3akJaWht/vZ8CAAV5Hct2uXbto2bIlxpiQMabaGPPWkbZvUvm88MILhEIhQqEQOTk5FBQUNGV3MWnAgAHs3r2bUCjEySefzJVXXul1JNfl5uYyd+5cWrdu7XUUV9XW1jJ+/HgWLlwIwLJlyygpKfE4VfPp0MFe96mthbw8+Oyz+rcrLCxk1qxZ7oaLMmlpaUydOpWamho2btxIaWlpXH0vNESrVq34+OOPcRwnALQBfmaMuf5w2zepfNq3bx/+d0VFBSYBp8gdM2ZM+C++nJwctm/f7nEi9/Xp04fevXt7HcN1M2bMoHXr1uTk5ACQnZ1NUVGRt6Ga2U9+AosWwe7dtoC2bj10m8LCQk499VT3w0WRzMxMBg8eDEC7du1IT09n3bp1Hqdyl8/n48QTT/zuYQtsvxz2XHSTl5S6+OKLSU5OZtmyZcyZM6epu4tpM2bMSMhfwolqw4YNpKenhx+ffvrpbK3vt3OM69rVDr3ets2egisr8zpRdFuxYgVlZWUJeSaotrYWY0wI2AGsdRxn+uG2PWr5GGO+3n/+7uC3BwDeeecd9u7dS7du3bj55pub77OIIscddxx+v/+Qt7Fjx4a36d69O0lJSXH3l+93GvI1SDTxNMDgaC68EEpLYeNGewPqrl1eJ4pO27Zto0ePHowaNeqAM0OJIjU19bvTbqcBZxpjLj/ctkedWNRxnPQjPB3+zXPvvfeSl5d3LDljRnl5+RGfHz58OH//+9/ZtGkTvjhdn/hoX4NEdNZZZx0wwGLTpk2cdNJJHiaKrG7dYO5cOxdc796wcCH84Adep4oeVVVVZGRk0L17dyZNmuR1HE85jrPZGLMauB6YV982TfpNuWjRovC/f//733P88cc3ZXcxaeLEicycOZNVq1Yl5OefyIYOHcru3btZvnw5AMuXL2fEiBHh578bCZeSklLvSLhvvvmGU045hZSUFFq2bMmKFStcy95YvXrBSy/Bu+/Cf/+3nRdOoK6ujrPPPpuTTz6Z+fPnex3HE+vXrw+P9jXGtAF+Chx+CLTjOI1+a9eunZOWlub4/X6nbdu2zrvvvuskmuTkZCcpKcnx+/2O3+93OnXq5HUk140ePdrx+XwO4BhjnPT0dK8juWbcuHFOSkqKAzj5+fnh99fU1DjJycnO0qVLnT179jh+v9+ZP3/+Aa8dNGhQ+PulsLDQOfnkk13N3hTFxY5jjOP07Ok4J598Svj/v8/nc4YNG+Z1PNcVFRU5QPj3gN/vd8aNG+d1LFe9/PLLTiAQcIAQUA0sdo7QH7rJVKQZHHyT6bRp0xg7diw7duwAoOf+mTrfeOON8DbHH388Dz74IDfeeCPV1dW0aNGCvXv3xsyp2+nT4frr4fLL7dFQSorXiSRKHDDs2RhzC9ALuMFxnPBwldj4LheJMQ0ZCVdRUUHXrl0B8Pv9JCUl8dFHH7masymuu84uyT1vHgwbBvv2eZ1IotR7QFdgmzGm1BhzAmglU5GIaOwZhVg56vnOyJFQWQl33WXvB9LRj2zZcuBjx3FWAafsH/n2J2C7Meb/Nem0W1JSklNXV9eUnCIiknhqm/Rnlt/vb9KAhXh4a9GihecZvH7T18D+Aff9x6FQKHzz9Z49e/D7/cybN++AbQYOHEinTp1wHIfCwkLat2/v+eeh7wN9DZr6BlQ5jmO+ewN+A9QBnwH99r8/TafdRCLA7/dzzz33kJ+fj+M45OTk0K9fP7Kzs8nKyuKBBx5g2rRpZGRkkJKSQlpaGqWlpV7HFomEpcAvHMd5/fvvbNJpt2Aw6FRWVjY1WEwLBoPoa6CvQbwtqdAY+j7Q1wDAGFPlOE7waNs16bRb//79m/LyuKCvgb4GYun7QF+D/f7akI10n49IM9CRj0hY5BeTExERaQyVj4iIuE7lIyIirlP5iIiI61Q+IiLiOpWPiIi4TuUjIiKuU/mIiIjrVD4iIuI6lY+IiLhO5SMiIq5T+YiIiOtUPiIi4jqVj4iIuE7lIyIirlP5iIiI61Q+IiLiOpWPiIi4TuUjIiKuU/mIiIjrVD4iIuI6lY+IiLhO5SMiIq5T+Yi47JNPPiE9PZ3U1FTS09PZtGlTvdsZYwgEAgQCAU488USXU4pElnEcpymvb9KLReKFMYaG/iydf/75HHfccbz++uv06tWLnTt3snLlyibtUySKmAZtpPIRabpjKYrU1FRWrVpFZmYma9as4fzzz6e2trZJ+xSJIg0qH51p2UFiAAAC+klEQVR2E3HZt99+S2ZmJgCZmZns3bv3sNsGg0FatmzJmDFj3Ion4gqVj0gEHHfccfj9/kPexo4d2+B9vPfee1RWVlJaWsojjzzCkiVLIphYxF0qH5FGKigoIBgMEgwGD3muvLyc6urqQ94efPBBUlJSWLNmDQBr1qwhOTm53v2fe+65AOTk5HDaaafx6quvRu6TEXGZrvmININjuT5z3nnnkZ6eHh5wUF5ezqpVqw7YZtOmTaSnp9OqVSs2bNjA2WefzZw5c+jbt28k4os0J13zEYlGxcXFrFq1Kjzw4IUXXgBg5syZnHnmmQAsXLiQtm3bEggE6NKlC9dcc42KR+KKjnxEmoFGpomE6chHRESik8pHRERcp/IRERHXqXxERMR1Kh8REXGdykdERFyn8hEREdepfERExHUqHxERcZ3KR0REXKfyERER16l8RETEdSofERFxncpHRERcp/IRERHXqXxERMR1Kh8REXGdykdERFyn8hEREdepfERExHUqHxERcZ3KR0REXKfyERER16l8RETEdSofERFxncpHxGWjRo3C7/djjGHmzJmH3W7ixImkpqaSkpJCr169XEwoEnkqHxGX5ebmMnfuXFq3bn3YbWpraxk/fjwLFy5k586dLFu2jJKSEhdTikRWstcBRBJNnz59jrrNjBkzaN26NTk5OQBkZ2dTVFRE3759I5xOxB068hGJQhs2bCA9PT38+PTTT2fr1q0eJhJpXiofkQg47rjj8Pv9h7yNHTu2Qa93HCfCCUW8ZfRNLtI4xphZwID9D/2O4yQd4+t3AYWO48yq57nhwEOO4xy///HrAI7jaOSBxAUd+Yg0kuM4QxzHCe5/O6biaYCZQGtjTJYxJgh0A55s5o8h4hmVj4jLjDG/M8bsBVoDM4wxO/a/v6sx5isAx3FqgAnAYmAX8LbjOBruJnFDp91ERMR1OvIRERHXqXxERMR1Kh8REXGdykdERFyn8hEREdepfERExHUqHxERcZ3KR0REXPf/AX8PL/Pz9UvvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt  #导入matplotlib库\n",
    "import numpy as np  #导入numpy库\n",
    "import mpl_toolkits.axisartist as axisartist#建立坐标轴\n",
    "%matplotlib inline\n",
    "\n",
    "#创建画布\n",
    "fig = plt.figure(figsize=(7, 5))\n",
    "#使用axisartist.Subplot方法创建一个绘图区对象ax\n",
    "ax = axisartist.Subplot(fig, 111)  \n",
    "#将绘图区对象添加到画布中\n",
    "fig.add_axes(ax)\n",
    "\n",
    "#绘制坐标系\n",
    "ax.axis[:].set_visible(False)\n",
    "ax.axis[\"x\"] = ax.new_floating_axis(0,0)\n",
    "ax.axis[\"x\"].set_axisline_style(\"->\", size = 1.0)\n",
    "ax.axis[\"y\"] = ax.new_floating_axis(1,0)\n",
    "ax.axis[\"y\"].set_axisline_style(\"->\", size = 1.0)\n",
    "ax.axis[\"x\"].set_axis_direction(\"top\")\n",
    "ax.axis[\"y\"].set_axis_direction(\"right\")\n",
    "\n",
    "x = np.arange(-3,4)\n",
    "y=1-x\n",
    "y[y<=0]=0\n",
    "plt.xlim(-3,3)\n",
    "plt.ylim(-1,3)\n",
    "plt.plot(x,y, c='b')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 什么是支持向量？为什么SVM中只有一小部分的训练样本是支持向量（稀疏的）？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "支持向量就是在分类问题中距离划分面最近的样本，落在最大间隔上\n",
    "在实际问题中，大多数样本都分布在距离划分面最近的样本构成的分隔面之后，并没有落在间隔面上，同时很难达到所有样本都无误的分布在正确区间。所以我们说SVM中实际上只要很小一部分样本是支持向量。\n",
    "在SVM的对偶问题中，对应αi>0的点，都是支持向量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 决策树中特征分裂的准则有哪些？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "分裂属性就是在某个节点处按照某一特征属性的不同划分构造不同的分支，其目标是让各个分裂子集尽可能地“纯”。尽可能“纯”就是尽量让一个分裂子集中待分类项属于同一类别。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "分裂属性分为三种不同的情况：\n",
    "1、属性是离散值且不要求生成二叉决策树。此时用属性的每一个划分作为一个分支。\n",
    "2、属性是离散值且要求生成二叉决策树。此时使用属性划分的一个子集进行测试，按照“属于此子集”和“不属于此子集”分成两个分支。\n",
    "3、属性是连续值。此时确定一个值作为分裂点split_point，按照>split_point和<=split_point生成两个分支。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***属性选择度量是一种选择分裂准则，是将给定的类标记的训练集合的数据划分D“最好”地分成个体类的启发式方法，它决定了拓扑结构及分裂点split_point的选择。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ID3算法：根据信息增益来决定分裂属性和分裂点，信息增益表示数据分裂后比分裂前信息熵的增加量。选取信息增益最大的点作为划分准则   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ID3算法存在一个问题，就是偏向于多值属性，例如，如果存在唯一标识属性ID，则ID3会选择它作为分裂属性，这样虽然使得划分充分纯净，但这种划分对分类几乎毫无用处。ID3的后继算法C4.5使用增益率（gain ratio）的信息增益扩充，试图克服这个偏倚。 同样选取信息增益最大的点作为划分准则。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM模型并没有概率解释。为了使SVM模型能输出概率，我们应该设置哪个参数？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在sklearn中，有三种SVM的调用方式，分别是：LinearSVC,SVC和SGDClassifier中的SVM linear_model。数据量特别大时选择SGDClassifier，数据量相对大时选择LinearSVC，数据量少时需要好的回归效果用SVM，而只有SVC提供了使用概率作为SVM评估标准的方法，其对模型计算出的置信度使用逻辑回归计算其概率。在sklearn.svm.SVC方法中，将probability参数设置为True。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ps.一些差别："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LinearSVC基于liblinear，罚函数是对截矩进行惩罚；SVC基于libsvm，罚函数不是对截矩进行惩罚。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LinearSVC使用的是平方hinge loss，SVC使用的是绝对值hinge loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LinearSVC使用的是One-vs-All（也成One-vs-Rest）的分类策略，而SVC可以使用One-vs-One和One-vs-All。分类策略决定了模型个数，ovo是n*（n-1）/2个，ova是n个"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SVM解决问题时，问题是分为线性可分和线性不可分问题的，liblinear对线性可分问题做了优化，故在大量数据上收敛速度比libsvm快（所以大规模数据的情况下适合LinearSVC）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 采用train_test_split，从将数据集中随机抽取10000条记录，用于下述作业中模型的训练（原始数据集太大，剩余数据抛弃）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'number')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZUAAAELCAYAAAARNxsIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAG2tJREFUeJzt3XuUXnV97/H3h0RQqpBgBo25OFEHakCkMGIUEQEbEk5L6Cq0QWtyMDWrGMC7wqFHWGBaqHpQWoQTTSRYFyHiJfEYDSkXUUuAAUIuBJoxIBmCZDAhUCnQwPf8sX8jO8MzmSfD75n9DPm81nrW7P3dv71/3z2ZzHd++6qIwMzMLIe9qk7AzMxeOVxUzMwsGxcVMzPLxkXFzMyycVExM7NsXFTMzCwbFxUzM8vGRcXMzLJxUTEzs2yGV53AYBs1alS0trZWnYaZ2ZBy1113PR4RLf212+OKSmtrKx0dHVWnYWY2pEj6TT3tfPjLzMyycVExM7NsXFTMzCwbFxUzM8vGRcXMzLJxUTEzs2xcVMzMLJuGFRVJCyRtkbS2V/xsSQ9IWifpn0rx8yR1pmUnluJTUqxT0rml+ARJt0vaIOk6SXs3al/MzKw+jRypXA1MKQckHQdMAw6LiEOAr6T4RGA6cEha5xuShkkaBlwBTAUmAqentgCXApdFRBuwDZjVwH0xM7M6NOyO+oi4VVJrr/CZwCUR8WxqsyXFpwGLUvxBSZ3AUWlZZ0RsBJC0CJgmaT1wPPCh1GYhcCFwZWP2ZnA9fNE7Br3P8V9cM+h9mtkrz2CfUzkIOCYdtvq5pHel+BhgU6ldV4r1FX898ERE7OgVr0nSbEkdkjq6u7sz7YqZmfU22EVlODASmAR8DlgsSYBqtI0BxGuKiHkR0R4R7S0t/T4PzczMBmiwHyjZBfwgIgK4Q9ILwKgUH1dqNxbYnKZrxR8HRkgankYr5fZmZlaRwR6p/IjiXAiSDgL2pigQS4HpkvaRNAFoA+4A7gTa0pVee1OczF+aitLNwKlpuzOBJYO6J2Zm9hING6lIuhb4ADBKUhdwAbAAWJAuM34OmJkKxDpJi4H7gB3AnIh4Pm3nLGA5MAxYEBHrUhdfABZJ+hJwDzC/UftiZmb1aeTVX6f3sehv+mg/F5hbI74MWFYjvpEXrxAzM7Mm4DvqzcwsGxcVMzPLxkXFzMyycVExM7NsXFTMzCwbFxUzM8vGRcXMzLJxUTEzs2xcVMzMLBsXFTMzy8ZFxczMsnFRMTOzbFxUzMwsGxcVMzPLxkXFzMyycVExM7NsGlZUJC2QtCW95bH3ss9KCkmj0rwkXS6pU9JqSUeU2s6UtCF9ZpbiR0pak9a5XJIatS9mZlafRo5Urgam9A5KGgf8KfBwKTyV4r30bcBs4MrU9gCK1xC/m+ItjxdIGpnWuTK17VnvJX2ZmdngauTrhG+V1Fpj0WXA54Elpdg04Jr0vvqVkkZIGk3xjvsVEbEVQNIKYIqkW4D9IuK2FL8GOAX4aWP2xmzomvs3p1bS7/n/en0l/Vq1BvWciqSTgUci4t5ei8YAm0rzXSm2q3hXjbiZmVWoYSOV3iTtC5wPTK61uEYsBhDvq+/ZFIfKGD9+fL+5mpnZwAzmSOWtwATgXkkPAWOBuyW9kWKkMa7UdiywuZ/42BrxmiJiXkS0R0R7S0tLhl0xM7NaBq2oRMSaiDgwIlojopWiMBwREb8FlgIz0lVgk4DtEfEosByYLGlkOkE/GVielj0laVK66msGO5+jMTOzCjTykuJrgduAgyV1SZq1i+bLgI1AJ/BN4OMA6QT9xcCd6XNRz0l74EzgW2mdX+OT9GZmlWvk1V+n97O8tTQdwJw+2i0AFtSIdwCHvrwszcwsJ99Rb2Zm2biomJlZNi4qZmaWjYuKmZll46JiZmbZuKiYmVk2LipmZpaNi4qZmWXjomJmZtm4qJiZWTYuKmZmlo2LipmZZeOiYmZm2biomJlZNi4qZmaWjYuKmZll46JiZmbZNPJ1wgskbZG0thT7sqT7Ja2W9ENJI0rLzpPUKekBSSeW4lNSrFPSuaX4BEm3S9og6TpJezdqX8zMrD6NHKlcDUzpFVsBHBoRhwH/AZwHIGkiMB04JK3zDUnDJA0DrgCmAhOB01NbgEuByyKiDdgGzGrgvpiZWR0aVlQi4lZga6/YDRGxI82uBMam6WnAooh4NiIeBDqBo9KnMyI2RsRzwCJgmiQBxwPXp/UXAqc0al/MzKw+VZ5T+Sjw0zQ9BthUWtaVYn3FXw88USpQPXEzM6tQJUVF0vnADuC7PaEazWIA8b76my2pQ1JHd3f37qZrZmZ1GvSiImkm8GfAhyOipxB0AeNKzcYCm3cRfxwYIWl4r3hNETEvItojor2lpSXPjpiZ2UsMalGRNAX4AnByRDxdWrQUmC5pH0kTgDbgDuBOoC1d6bU3xcn8pakY3QycmtafCSwZrP0wM7PaGnlJ8bXAbcDBkrokzQL+BXgdsELSKklXAUTEOmAxcB/wM2BORDyfzpmcBSwH1gOLU1soitOnJXVSnGOZ36h9MTOz+gzvv8nARMTpNcJ9/uKPiLnA3BrxZcCyGvGNFFeHmZlZk/Ad9WZmlo2LipmZZeOiYmZm2biomJlZNi4qZmaWjYuKmZll46JiZmbZuKiYmVk2LipmZpaNi4qZmWXjomJmZtm4qJiZWTYuKmZmlo2LipmZZeOiYmZm2biomJlZNi4qZmaWTSNfJ7xA0hZJa0uxAyStkLQhfR2Z4pJ0uaROSaslHVFaZ2Zqv0HSzFL8SElr0jqXS1Kj9sXMzOrTyJHK1cCUXrFzgRsjog24Mc0DTAXa0mc2cCUURQi4AHg3xauDL+gpRKnN7NJ6vfsyM7NB1rCiEhG3Alt7hacBC9P0QuCUUvyaKKwERkgaDZwIrIiIrRGxDVgBTEnL9ouI2yIigGtK2zIzs4oM9jmVN0TEowDp64EpPgbYVGrXlWK7infViNckabakDkkd3d3dL3snzMystmY5UV/rfEgMIF5TRMyLiPaIaG9paRlgimZm1p/BLiqPpUNXpK9bUrwLGFdqNxbY3E98bI24mZlVqN+iImmYpH/L1N9SoOcKrpnAklJ8RroKbBKwPR0eWw5MljQynaCfDCxPy56SNCld9TWjtC0zM6vI8P4aRMTzkp6WtH9EbK93w5KuBT4AjJLURXEV1yXAYkmzgIeB01LzZcBJQCfwNHBG6nurpIuBO1O7iyKi5+T/mRRXmL0G+Gn6mJlZhfotKskzwBpJK4Df9wQj4py+VoiI0/tYdEKNtgHM6WM7C4AFNeIdwKG7TtvMzAZTvUXlJ+ljZmbWp7qKSkQslPQaYHxEPNDgnMzMbIiq6+ovSX8OrAJ+luYPl7S0kYmZmdnQU+8lxRdSPCblCYCIWAVMaFBOZmY2RNVbVHbUuPKrz5sNzcxsz1Tvifq1kj4EDJPUBpwD/Hvj0jIzs6Go3pHK2cAhwLPAtcCTwCcblZSZmQ1N9V799TRwvqRLi9l4qrFpmZnZUFTv1V/vkrQGWE1xE+S9ko5sbGpmZjbU1HtOZT7w8Yj4BYCk9wHfBg5rVGJmZjb01HtO5ameggIQEb8EfAjMzMx2ssuRSuld8XdI+r8UJ+kD+GvglsamZmZmQ01/h7++2mv+gtK071MxM7Od7LKoRMRxg5WImZkNfXWdqJc0guJFWK3ldXb16Hszs6Hmwgsv3KP6bYR6r/5aBqwE1gAvNC4dMzMbyuotKq+OiE/n6lTSp4C/pTgvs4biTY+jgUXAAcDdwEci4jlJ+wDXAEcCvwP+OiIeSts5D5gFPA+cExHLc+VoZma7r95Lir8j6WOSRks6oOczkA4ljaF4dlh7RBwKDAOmA5cCl0VEG7CNoliQvm6LiLcBl6V2SJqY1jsEmAJ8Q9KwgeRkZmZ51FtUngO+DNwG3JU+HS+j3+HAayQNB/YFHgWOB65PyxcCp6TpaWmetPwESUrxRRHxbEQ8SPF++6NeRk5mZvYy1Xv469PA2yLi8ZfbYUQ8IukrwMPAfwE3UBSpJyJiR2rWBYxJ02OATWndHZK2A69P8ZWlTZfXMTOzCtRbVNYBT+foUNJIilHGBIqXfn0PmFqjac99MOpjWV/xWn3OBmYDjB8/fjczNoCj//noSvr91dm/qqRfMxuYeovK88AqSTdTPP4eGPAlxR8EHoyIbgBJPwDeC4yQNDyNVsYCm1P7LmAc0JUOl+0PbC3Fe5TX2UlEzAPmAbS3t/umTTOzBqn3nMqPgLkUL+a6q/QZiIeBSZL2TedGTgDuA24GTk1tZgJL0vTSNE9aflNERIpPl7SPpAlAG3DHAHMyM7MM6n2fysL+W9UnIm6XdD3FZcM7gHsoRhE/ARZJ+lKKzU+rzKe4+qyTYoQyPW1nnaTFFAVpBzAnIp7PlaeZme2+eu+of5Aa5ysi4i0D6TQiLmDn54gBbKTG1VsR8QxwWh/bmUsxgjIzsyZQ7zmV9tL0qyl+yQ/oPhUzM3vlquucSkT8rvR5JCK+RnFfiZmZ2R/Ue/jriNLsXhQjl9c1JCMzMxuy6j389VVePKeyA3iIPs5zmJnZnqveojIV+Et2fvT9dOCiBuRkZmZDVL1F5UcUd7/fDTzTuHTMzGwoq7eojI2IKQ3NxMzMhrx676j/d0nvaGgmZmY25NU7Unkf8D/TTZDPUjzMMSLisIZlZmZmQ87unKg3MzPbpXqf/fWbRidiZmZDX73nVMzMzPrlomJmZtm4qJiZWTYuKmZmlo2LipmZZeOiYmZm2VRSVCSNkHS9pPslrZf0HkkHSFohaUP6OjK1laTLJXVKWl1+DL+kman9Bkkz++7RzMwGQ1Ujla8DP4uIPwbeCawHzgVujIg24MY0D8WNl23pMxu4EkDSARSvJH43xWuIL+gpRGZmVo1BLyqS9gPeD8wHiIjnIuIJYBqwMDVbCJySpqcB10RhJTBC0mjgRGBFRGyNiG3ACsAPvTQzq1AVI5W3AN3AtyXdI+lbkv4IeENEPAqQvh6Y2o8BNpXW70qxvuJmZlaRKorKcOAI4MqI+BPg97x4qKsW1YjFLuIv3YA0W1KHpI7u7u7dzdfMzOpURVHpAroi4vY0fz1FkXksHdYifd1Saj+utP5YYPMu4i8REfMioj0i2ltaWrLtiJmZ7WzQi0pE/BbYJOngFDoBuA9YCvRcwTUTWJKmlwIz0lVgk4Dt6fDYcmCypJHpBP3kFDMzs4rU++j73M4Gvitpb2AjcAZFgVssaRbwMHBaarsMOAnoBJ5ObYmIrZIuBu5M7S6KiK2DtwtmZtZbJUUlIlYB7TUWnVCjbQBz+tjOAmBB3uzMzGygfEe9mZll46JiZmbZuKiYmVk2LipmZpaNi4qZmWXjomJmZtm4qJiZWTYuKmZmlk1Vd9SbmVkdFn/vqEr6/avT7hjQeh6pmJlZNi4qZmaWjYuKmZll46JiZmbZuKiYmVk2LipmZpaNi4qZmWXjomJmZtlUVlQkDZN0j6T/l+YnSLpd0gZJ16VXDSNpnzTfmZa3lrZxXoo/IOnEavbEzMx6VDlS+QSwvjR/KXBZRLQB24BZKT4L2BYRbwMuS+2QNBGYDhwCTAG+IWnYIOVuZmY1VFJUJI0F/gfwrTQv4Hjg+tRkIXBKmp6W5knLT0jtpwGLIuLZiHgQ6ASqeZ6BmZkB1Y1UvgZ8Hnghzb8eeCIidqT5LmBMmh4DbAJIy7en9n+I11jHzMwqMOhFRdKfAVsi4q5yuEbT6GfZrtbp3edsSR2SOrq7u3crXzMzq18VI5WjgZMlPQQsojjs9TVghKSepyaPBTan6S5gHEBavj+wtRyvsc5OImJeRLRHRHtLS0vevTEzsz8Y9KISEedFxNiIaKU40X5TRHwYuBk4NTWbCSxJ00vTPGn5TRERKT49XR02AWgDBvasZjMzy6KZ3qfyBWCRpC8B9wDzU3w+8B1JnRQjlOkAEbFO0mLgPmAHMCcinh/8tM3MrEelRSUibgFuSdMbqXH1VkQ8A5zWx/pzgbmNy9DMzHaH76g3M7NsXFTMzCwbFxUzM8vGRcXMzLJxUTEzs2xcVMzMLBsXFTMzy8ZFxczMsnFRMTOzbJrpMS1mtgdZP/emQe/z7ecfP+h97mk8UjEzs2w8UrEh6+fvP7aSfo+99ed9LvuXz/x4EDN50Vlf/fNK+jXrzSMVMzPLxkXFzMyycVExM7NsXFTMzCybQS8qksZJulnSeknrJH0ixQ+QtELShvR1ZIpL0uWSOiWtlnREaVszU/sNkmb21aeZmQ2OKkYqO4DPRMTbgUnAHEkTgXOBGyOiDbgxzQNMpXj/fBswG7gSiiIEXAC8m+KNkRf0FCIzM6vGoBeViHg0Iu5O008B64ExwDRgYWq2EDglTU8DronCSmCEpNHAicCKiNgaEduAFcCUQdwVMzPrpdJzKpJagT8BbgfeEBGPQlF4gANTszHAptJqXSnWV9zMzCpSWVGR9Frg+8AnI+LJXTWtEYtdxGv1NVtSh6SO7u7u3U/WzMzqUskd9ZJeRVFQvhsRP0jhxySNjohH0+GtLSneBYwrrT4W2JziH+gVv6VWfxExD5gH0N7e/ofCc+TnrnnZ+zIQd315RiX9mpk1WhVXfwmYD6yPiP9TWrQU6LmCayawpBSfka4CmwRsT4fHlgOTJY1MJ+gnp5iZmVWkipHK0cBHgDWSVqXY/wIuARZLmgU8DJyWli0DTgI6gaeBMwAiYquki4E7U7uLImLr4OyCmZnVMuhFJSJ+Se3zIQAn1GgfwJw+trUAWJAvOzMzezl8R72ZmWXjomJmZtm4qJiZWTYuKmZmlo2LipmZZeOiYmZm2biomJlZNi4qZmaWjYuKmZll46JiZmbZuKiYmVk2LipmZpaNi4qZmWXjomJmZtm4qJiZWTYuKmZmlo2LipmZZTPki4qkKZIekNQp6dyq8zEz25MN6aIiaRhwBTAVmAicLmlitVmZme25hnRRAY4COiNiY0Q8BywCplWck5nZHmuoF5UxwKbSfFeKmZlZBRQRVecwYJJOA06MiL9N8x8BjoqIs3u1mw3MTrMHAw9k6H4U8HiG7eTWjHk5p/o4p/o1Y16v9JzeHBEt/TUanqmzqnQB40rzY4HNvRtFxDxgXs6OJXVERHvObebQjHk5p/o4p/o1Y17OqTDUD3/dCbRJmiBpb2A6sLTinMzM9lhDeqQSETsknQUsB4YBCyJiXcVpmZntsYZ0UQGIiGXAsgq6zno4LaNmzMs51cc51a8Z83JODPET9WZm1lyG+jkVMzNrIi4qZmaWzR5ZVCS9UdIiSb+WdJ+kZZIOkrS2wf2eJmmdpBcktfdaVlVOX5Z0v6TVkn4oaUQT5HRxymeVpBskvanX8kryKvX/WUkhaVTVOUm6UNIj6Xu1StJJVeeU+j47PZNvnaR/qjonSdeVvkcPSVrVBDkdLmllyqlD0lG9lleV1zsl3SZpjaQfS9pvtzYQEXvUBxBwG/B3pdjhwDHA2gb3/XaKmy9vAdqbJKfJwPA0fSlwaRPktF9p+hzgqmb4XqW+xlFcbfgbYFTVOQEXAp+tEa8yp+OAfwP2SfMHVp1Tr/y+Cnyx6pyAG4Cpafok4JYm+fe7Ezg2TX8UuHh31t8TRyrHAf8dEVf1BCJiFaXHvUhqlfQLSXenz3tTfLSkW9NfFmslHSNpmKSr0/waSZ/qq+OIWB8Rte7mrzKnGyJiR5pdSXEDadU5PVma/SOgfDVJZXkllwGfb7KcaqkypzOBSyLi2dTvlibIqWf7Av4KuLYJcgqgZxSwPzvfuF1lXgcDt6bpFcBf7qLtSwz5S4oH4FDgrn7abAH+NCKekdRG8QPYDnwIWB4Rc1U8IXlfir8exkTEoQAqHT4agjl9FLiuGXKSNBeYAWyn+A/Wo7K8JJ0MPBIR9xa/m6rPKTlL0gygA/hMRGyrOKeDgGPSv+EzFCOpOyvOqccxwGMRsSHNV5nTJ4Hlkr5CcSrivaVlVea1FjgZWAKcxs5PLenXnjhSqcergG9KWgN8j+Kx+lAMC8+QdCHwjoh4CtgIvEXSP0uaAjxZa4PNnpOk84EdwHebIaeIOD8ixqV8ztqNnBqSl6R9gfOBL+5mLg3LKbkSeCvFL41HKQ7tVJ3TcGAkMAn4HLBYvapwBTn1OJ0XRyn1alROZwKfSj/nnwLmN0leHwXmSLoLeB3w3G5l1chjc834AU4Abq0RbyUdq6Q4Tt3z18NwYEep3ZuAjwFrgBkp9lqKIeKPKe7q7y+HW9j5nEqlOQEzKY7f7tssOZW282ZKx5Crygt4B8Vfhg+lzw7gYeCNTfS9KvdXWU7Az4APlOZ/DbRU/X1K23sMGFv1z1Nqt50X7xUU8GQz5NWrv4OAO+pp2/PZE0cqNwH7SPpYT0DSuyh+efXYH3g0Il4APkLxCBgkvRnYEhHfpPir4ggVVwDtFRHfB/43cMRQyin91fIF4OSIeLpJcmorzZ4M3F91XhGxJiIOjIjWiGileJjpERHx24q/V6NLs39Bceiisu9T8iPg+LStg4C9KZ6UW/X/vQ8C90dEVylWZU6bgWPT9PHAhtKyKn+mDkxf9wL+Hriqr7Y17U4FeqV8KKr4Yoq/oNYBPwHaePEvgDZgNcWJ638E/jPFZ1L8p70H+AUwAXgncDewKn2m7qLfv6D4ZfQsxV9My5sgp06Kk389ba9qgpy+n9ZfTfFX1Zhm+PfrlcNDpKu/Kv5efYfir9HVFA9THd0EOe0N/Gvaxt3A8VXnlLZxNaWrqarOCXgfxXmTe4HbgSObJK9PAP+RPpeQRlP1fvyYFjMzy2ZPPPxlZmYNsideUtxwkq4Aju4V/npEfLuKfMA57Y5mzMs51cc51a9Refnwl5mZZePDX2Zmlo2LipmZZeOiYpaRpBGSPl51HmZVcVExy2sEUHdRUeFl/T+U5AturGn4h9Esr0uAt6p4X8fNwGEUz8F6FfD3EbFEUivw07T8PcApkj5I8WSDzRR3Vj8bEWdJaqG4o3l82v4nI+JX6blOb6J4bMfjFA8RNKuci4pZXucCh0bE4WkEsW9EPJkek7FS0tLU7mDgjIj4uIqXkPU8OuMpikd03JvafR24LCJ+KWk8xbtc3p6WHQm8LyL+a3B2zax/LipmjSPgHyS9H3gBGAO8IS37TUSsTNNHAT+PiK0Akr5H8SA/KJ5XNbH0kN/9JL0uTS91QbFm46Ji1jgfpng675ER8d+SHgJenZb9vtRuV4+F3wt4T+/ikYrM72uuYVYhn6g3y+spindQQPEU2S2poBzHzk+YLbsDOFbSyHTIrPymvRsovU9G0uENyNksG49UzDKKiN9J+pWktRQvS/pjSR0UT4a9v491HpH0DxRPqt0M3Efxrg2Ac4ArJK2m+P96K/B3Dd4NswHzY1rMmoCk10bEf6aRyg8pXqL0w6rzMttdPvxl1hwuTJchrwUepHjRldmQ45GKmZll45GKmZll46JiZmbZuKiYmVk2LipmZpaNi4qZmWXjomJmZtn8f6hs7Hva5FFpAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_data=pd.read_csv(\"D:\\\\python\\\\csdn data\\\\otto\\\\otto_train.csv\")\n",
    "train_target=train_data[\"target\"]\n",
    "sns.countplot(train_target)\n",
    "plt.xlabel(\"targer\")\n",
    "plt.ylabel(\"number\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wangcc\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_split.py:2026: FutureWarning: From version 0.21, test_size will always complement train_size unless both are specified.\n",
      "  FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "X=train_data.iloc[:,1:-1]\n",
    "y=train_data.iloc[:,-1]\n",
    "x_train,x_test,y_train,y_test=train_test_split(X,y,train_size=10000/len(X),stratify=y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通常在类分布不平衡的情况下会用到stratify参数，用以保持split前类的分布，防止因随机抽取造成分类的失衡\n",
    "根据上图可以观察出9种分类并不均衡，随机抽取样本数量减少很可能出现某类别数目失衡。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 93)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "print(x_train.shape)\n",
    "print(y_train.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "分类完成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 根据6中得到的训练数据，训练线性SVM，并对超参数（正则惩罚项、C）进行超参数调优，并分析不同C对应的训练误差和交叉验证得到的测试误差的变化趋势。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 5 candidates, totalling 25 fits\n",
      "[CV] C=0.1 ...........................................................\n",
      "[CV] .................. C=0.1, score=0.5921118322516226, total=   9.3s\n",
      "[CV] C=0.1 ...........................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   14.6s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .................. C=0.1, score=0.6003996003996004, total=   9.7s\n",
      "[CV] C=0.1 ...........................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   29.9s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .................. C=0.1, score=0.6026986506746627, total=   9.7s\n",
      "[CV] C=0.1 ...........................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:   45.2s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .................. C=0.1, score=0.5980980980980981, total=   9.6s\n",
      "[CV] C=0.1 ...........................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:  1.0min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] .................. C=0.1, score=0.5976953907815631, total=   9.6s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] ...................... C=1, score=0.72591113330005, total=   8.3s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .................... C=1, score=0.7282717282717283, total=   8.7s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .................... C=1, score=0.7376311844077961, total=   8.2s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .................... C=1, score=0.7407407407407407, total=   8.2s\n",
      "[CV] C=1 .............................................................\n",
      "[CV] .................... C=1, score=0.7414829659318637, total=   8.1s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ................... C=10, score=0.7299051422865701, total=   8.6s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ................... C=10, score=0.7382617382617382, total=   8.7s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] .................... C=10, score=0.750624687656172, total=   8.9s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ................... C=10, score=0.7402402402402403, total=   8.7s\n",
      "[CV] C=10 ............................................................\n",
      "[CV] ................... C=10, score=0.7545090180360722, total=   8.7s\n",
      "[CV] C=50 ............................................................\n",
      "[CV] ................... C=50, score=0.7338991512730904, total=   8.7s\n",
      "[CV] C=50 ............................................................\n",
      "[CV] ................... C=50, score=0.7312687312687313, total=   8.8s\n",
      "[CV] C=50 ............................................................\n",
      "[CV] ................... C=50, score=0.7326336831584208, total=   9.0s\n",
      "[CV] C=50 ............................................................\n",
      "[CV] ................... C=50, score=0.7357357357357357, total=   8.8s\n",
      "[CV] C=50 ............................................................\n",
      "[CV] ................... C=50, score=0.7434869739478958, total=   8.8s\n",
      "[CV] C=100 ...........................................................\n",
      "[CV] .................. C=100, score=0.7244133799301048, total=   8.8s\n",
      "[CV] C=100 ...........................................................\n",
      "[CV] .................. C=100, score=0.7247752247752248, total=   8.9s\n",
      "[CV] C=100 ...........................................................\n",
      "[CV] .................. C=100, score=0.7266366816591704, total=   8.8s\n",
      "[CV] C=100 ...........................................................\n",
      "[CV] .................. C=100, score=0.7277277277277278, total=   8.8s\n",
      "[CV] C=100 ...........................................................\n",
      "[CV] .................. C=100, score=0.7394789579158316, total=   8.8s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  25 out of  25 | elapsed:  5.8min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'mean_fit_time': array([8.34599924, 7.23127408, 7.59872284, 7.68423243, 7.70634055]), 'std_fit_time': array([0.1258771 , 0.22315124, 0.08994474, 0.10087133, 0.03703895]), 'mean_score_time': array([1.32730722, 1.18146677, 1.21520805, 1.20905819, 1.20370536]), 'std_score_time': array([0.02008363, 0.00401331, 0.00563874, 0.00454185, 0.00810949]), 'param_C': masked_array(data=[0.1, 1, 10, 50, 100],\n",
      "             mask=[False, False, False, False, False],\n",
      "       fill_value='?',\n",
      "            dtype=object), 'params': [{'C': 0.1}, {'C': 1}, {'C': 10}, {'C': 50}, {'C': 100}], 'split0_test_score': array([0.59211183, 0.72591113, 0.72990514, 0.73389915, 0.72441338]), 'split1_test_score': array([0.6003996 , 0.72827173, 0.73826174, 0.73126873, 0.72477522]), 'split2_test_score': array([0.60269865, 0.73763118, 0.75062469, 0.73263368, 0.72663668]), 'split3_test_score': array([0.5980981 , 0.74074074, 0.74024024, 0.73573574, 0.72772773]), 'split4_test_score': array([0.59769539, 0.74148297, 0.75450902, 0.74348697, 0.73947896]), 'mean_test_score': array([0.5982, 0.7348, 0.7427, 0.7354, 0.7286]), 'std_test_score': array([0.00353488, 0.0064752 , 0.00885124, 0.00429832, 0.00556649]), 'rank_test_score': array([5, 3, 1, 2, 4]), 'split0_train_score': array([0.62060773, 0.86770039, 0.9656121 , 0.98924597, 0.99562336]), 'split1_train_score': array([0.61290323, 0.86709177, 0.96624156, 0.98949737, 0.99612403]), 'split2_train_score': array([0.61745218, 0.86748344, 0.96624578, 0.99012377, 0.99462433]), 'split3_train_score': array([0.61672082, 0.86603349, 0.96513372, 0.99075231, 0.99550112]), 'split4_train_score': array([0.6111944 , 0.8661919 , 0.96489255, 0.98950525, 0.99425287]), 'mean_train_score': array([0.61577567, 0.8669002 , 0.96562514, 0.98982493, 0.99522514]), 'std_train_score': array([0.00335535, 0.0006738 , 0.00055561, 0.00054663, 0.00068542])}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "svc = SVC()\n",
    "param = {\"C\":[1e-1,1,10,50,100]}\n",
    "clf = GridSearchCV(svc,param,return_train_score=True,cv=5,verbose=5)\n",
    "clf = clf.fit(x_train,y_train)\n",
    "\n",
    "print(clf.cv_results_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "verbose：日志冗长度，int：冗长度，0：不输出训练过程，1：偶尔输出，>1：对每个子模型都输出。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最佳训练效果得分： 0.7483\n",
      "最佳超参数 {'C': 10}\n"
     ]
    }
   ],
   "source": [
    "print(\"最佳训练效果得分：\",clf.best_score_)\n",
    "print(\"最佳超参数\",clf.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'score')"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8lNX1x/HPMbLKKmAFIoIKuCBFG3GhVbQuoK2IthSqlqoVW3dtXah1KVardV9QRMWCFiniAv2BsigorVAJimyKLLUSsYIsKpsQcn5/3IkMIWQmIU+emeT7fr3mlZlnmTnR8Jy5z733XHN3REREyrJH3AGIiEjmU7IQEZGUlCxERCQlJQsREUlJyUJERFJSshARkZSULEREJCUlCxERSUnJQkREUtoz7gAqS/Pmzb1t27ZxhyEiklVmz579hbu3SHVctUkWbdu2JT8/P+4wRESyipn9N53jdBtKRERSUrIQEZGUIksWZjbMzFaa2fxd7Dcze9jMlpjZXDM7MmlffzNbnHj0jypGERFJT5R9Fn8FHgVG7GJ/T6B94nE08DhwtJntDdwK5AEOzDazce6+trwBbN26lYKCAjZv3lyB8LNL3bp1yc3NpVatWnGHIiLVUGTJwt3fMrO2ZRzSCxjhYUGNmWbWxMxaAt2Bye6+BsDMJgM9gOfLG0NBQQENGzakbdu2mFl5T88a7s7q1aspKCigXbt2cYcjItVQnH0WrYHlSa8LEtt2tb3cNm/eTLNmzap1ogAwM5o1a1YjWlAiEo84k0VpV3AvY/vOb2A2wMzyzSx/1apVpX9INU8UxWrK7yki8YhznkUBsF/S61xgRWJ79xLbp5X2Bu4+FBgKkJeXVznrw3ZPfPS0Uj9SRCReRUXw1Vewdm14rFsHtWvD978f6cfGmSzGAZeb2ShCB/eX7v6ZmU0E7jSzponjTgUGxhXk7lq3bh0jR47k0ksvLdd5p59+OiNHjqRJkyYRRSYisdmyJVzkiy/2yRf+0p4nv/7yS/AS340bNgwJJEKRJQsze57QQmhuZgWEEU61ANx9CDABOB1YAmwELkjsW2NmtwOzEm81qLizOxutW7eOxx57bKdksW3bNnJycnZ53oQJE6IOTUQqQ1ERfPwxfPABfPFFehf/jRvLfs+6daFJE2jaNDxatoRDDw3Pk7cXP99338h/zShHQ/VLsd+By3axbxgwLIq4qtqNN97I0qVL6dKlC7Vq1aJBgwa0bNmSOXPmsHDhQs466yyWL1/O5s2bueqqqxgwYACwvXzJ+vXr6dmzJ9///vd5++23ad26NWPHjqVevXox/2YiNdDatTBvXnjMnRse8+bBhg07H9uw4faLetOmcNBBO1/kd/W8bt2q/91SqDa1oVK6+mqYMyf1ccXHFPddlKVLF3jwwTIPueuuu5g/fz5z5sxh2rRpnHHGGcyfP//bIa7Dhg1j7733ZtOmTRx11FGcc845NGvWbIf3WLx4Mc8//zxPPvkkffr04cUXX+S8885LHZ+IVMzWrfDRR9sTQnFSWJ40ULNpU+jcGS68MPw89FDYZ5+wvXFj2LN6XV6r12+TBbp27brDXIiHH36Yl19+GYDly5ezePHinZJFu3bt6NKlCwDf+973+Pjjj6ssXpFqzR3+97/tyaA4MXzwQehXAKhVCw4+GI4/PiSFww8PP1u1gho0CrHmJIsULYBvRTwaaq+99vr2+bRp05gyZQozZsygfv36dO/evdS5EnXq1Pn2eU5ODps2bYokNpFqbeNGWLhwx5bC3Lmhn6FY69YhEZx2WvjZuTN07BhGG9VwNSdZxKRhw4Z8/fXXpe778ssvadq0KfXr1+fDDz9k5syZVRydSDVU3OFcsrWwePH2UUT160OnTnDWWdtbC4cfDiVa9bKdkkXEmjVrRrdu3ejUqRP16tXjO9/5zrf7evTowZAhQ+jcuTMdO3bkmGOOiTFSkSy0bt3Onc3z5sH69WG/GRx4YEgI/fptby0ccADsoaLb5WFecrxulsrLy/OSix998MEHHHLIIeV7oyyelFeh31ckGyR3OCcnh9I6nJMfhx0GSbd+ZWdmNtvd81Idp5ZFSVmYJESqjeIO5+SEULLDec894ZBDQodzcWdzDexwrmpKFiISj+QO5+TkULLD+fDD1eGcAZQsRCRaxR3OJfsWFi8O+2DHDufi1oI6nDOKkoWIVJ7kDufknyU7nA8/HPr2VYdzFlGyEJHyKywsfYbzJ59sP6a4w/mCC7a3FA47DBo0iC9uqTAlixKyeDCUSLT+/W8YOhTefTf0NZTscP7BD9ThXI0pWUSsoiXKAR588EEGDBhA/fr1I4hMJA3u8Oqr8Je/wJtvQk4ONGoUaq0VtxYOPlgdzjWAbhJGrLhEeUU8+OCDbExVylgkClu3wrPPhoRwxhmwdCncf3+ourpmDdx9N5x7btivRFEjqGURseQS5aeccgr77LMPo0eP5ptvvqF379788Y9/ZMOGDfTp04eCggK2bdvGzTffzOeff86KFSs48cQTad68OVOnTo37V5GaYP16eOqpkBiWLw99DMOHh85oJYUarcYki5gqlO9QonzSpEmMGTOGd955B3fnzDPP5K233mLVqlW0atWK8ePHA6FmVOPGjbn//vuZOnUqzZs3Tx2MyO5YuRIeeQQGDw6th+OPh8cfh9NPV7+DABHfhjKzHma2yMyWmNmNpezf38xeN7O5ZjbNzHKT9m0zszmJx7go46wqkyZNYtKkSRxxxBEceeSRfPjhhyxevJjDDz+cKVOmcMMNNzB9+nQaN24cd6hSUyxdCpdeCvvvD3fcEb4lzZgR+ifOOEOJQr4V5bKqOcBg4BSgAJhlZuPcfWHSYfcCI9x9uJmdBPwZOD+xb5O7d6mseDKhQrm7M3DgQC655JKd9s2ePZsJEyYwcOBATj31VG655ZbKD0Ck2OzZodN6zJgwmukXv4Df/S7MjhYpRZQti67AEndf5u5bgFFArxLHHAq8nng+tZT9WS+5RPlpp53GsGHDWJ+YoPTpp5+ycuVKVqxYQf369TnvvPP43e9+x7vvvrvTuSK7zR0mTYKTT4a8PHjtNbjuOvjPf+DJJ5UopExR9lm0BpJKQlIAHF3imPeBc4CHgN5AQzNr5u6rgbpmlg8UAne5+yslP8DMBgADANq0aVP5v0ElSC5R3rNnT37+859z7LHHAtCgQQOee+45lixZwnXXXccee+xBrVq1ePzxxwEYMGAAPXv2pGXLlurgloorLIQXXggtiTlzoGXL8HzAgLD8p0gaIitRbmY/BU5z918lXp8PdHX3K5KOaQU8CrQD3iIkjsPc/Usza+XuK8zsAOAN4IfuvnRXn1dZJcqzeVKeSpTLDjZuhGHD4L77Qm2mgw8OLYlzz4Wk1RelZsuEEuUFwH5Jr3OBFckHuPsK4GwAM2sAnOPuXybtw92Xmdk04Ahgl8mismRjkhDZwRdfhFFNjzwCq1fDsceGTrsf/1j1l6TCovzLmQW0N7N2ZlYb6AvsMKrJzJqbWXEMA4Fhie1NzaxO8TFANyC5Y1xESvr4Y7jyyjCy6bbb4LjjYPp0ePtt6NVLiUJ2S2QtC3cvNLPLgYlADjDM3ReY2SAg393HAd2BP5uZE25DXZY4/RDgCTMrIiS0u0qMoipPHFgNGP5XXVY8lAp4//3QB/H3v4eEcO65YWTTYYfFHZlUI5FOynP3CcCEEttuSXo+BhhTynlvA4fv7ufXrVuX1atX06xZs2qdMNyd1atXU7du3bhDkariDlOnhiQxcWKo5Hr11eGRm5v6fJFyqtYzuHNzcykoKGDVqlVxhxK5unXrkquLRPW3bRu89FJIEvn58J3vwJ13wq9/HUqCi0SkWieLWrVq0a5du7jDENl9mzaFGk333htmXR90EDzxRJhMpxalVIFqnSxEst7atfDYY/Dww6F+01FHhYqvZ50VyoWLVBElC5FMtHw5PPBAWGxowwbo2ROuvx5OOEH1miQWShYimWT+fLjnHhg5MnRi9+sXJtJ17hx3ZFLDKVmIxM0d/vnPcHtp/HioXx8uuwyuuSbMmRDJAEoWInEpKoKxY8PIppkzoXlzGDQolAxv1izu6ER2oGQhUtW++SYsWXrPPfDRR9CuXSjP8ctfhlaFSAZSshCpKl9+CUOGhDpN//sfHHEEjBoF55wT1pQQyWD6CxWJ2ooVIUEMGQJffw2nnBJaFj/8oUY2SdZQshCJyocfhltNzz4bZl736RNGNh15ZNyRiZSbkoVIZZsxI4xsGjsW6tULiwxdey0ccEDckYlUmJKFSGUoKgrDXv/ylzAMdu+94ZZb4PLLoUWLuKMT2W1KFiK7Y8uWMIHunntg4UJo0wYeegguugj22ivu6EQqjZKFSEV8/XUoxfHAA/Dpp2GG9XPPhX6JWrXijk6k0kW6dJaZ9TCzRWa2xMxuLGX//mb2upnNNbNpZpabtK+/mS1OPPpHGadI2j7/HH7/e9hvv7DAUIcO8OqrMGdOWHRIiUKqqchaFmaWAwwGTiGsxz3LzMaVWPHuXmCEuw83s5OAPwPnm9newK1AHuDA7MS5a6OKV6RMixeH8uDDh4dbT+ecEwr7HXVU3JGJVIkoWxZdgSXuvszdtwCjgF4ljjkUeD3xfGrS/tOAye6+JpEgJgM9IoxVpHSzZsFPfgIdO4ZE8ctfwqJF8MILShRSo0SZLFoDy5NeFyS2JXsfOCfxvDfQ0MyapXmuSDTcw62lE0+Erl3h9ddh4ED4+OMwsa59+7gjFKlyUSaL0qameonXvwNOMLP3gBOAT4HCNM/FzAaYWb6Z5deEpVMlYlu3wt/+Bl26wOmnw5IlcN998MkncMcdsO++cUcoEpsok0UBsF/S61xgRfIB7r7C3c929yOAmxLbvkzn3MSxQ909z93zWmgsu1TUhg1hJbqDDoLzzoPCQvjrX8PypddeCw0bxh2hSOyiTBazgPZm1s7MagN9gXHJB5hZczMrjmEgMCzxfCJwqpk1NbOmwKmJbSKVZ9UquPXWMDfiqqvC2hH/+AfMmwf9+0Pt2nFHKJIxIhsN5e6FZnY54SKfAwxz9wVmNgjId/dxQHfgz2bmwFvAZYlz15jZ7YSEAzDI3ddEFavUMMuWhdtLw4bB5s1hPevrr4djj407MpGMZe47dQVkpby8PM/Pz487DMlk774bynG88ALk5MAvfhHmShx8cNyRicTGzGa7e16q4zSDW6o39zCa6e67YcoUaNQoJIirroJWreKOTiRrKFlI9VRYCC++GFoS774LLVuGhHHJJdC4cdzRiWQdJQupXjZuDCOZ7rsv9E107AhPPRVGOdWpE3d0IllLyUKqh9Wr4bHHwhDYL76AY44JCePMM2GPSEugidQIShaS3f77X7j//tB62LgRfvSjMLLp+9/XkqUilUjJQrLT3LmhP2LUqJAUzj03LFl62GFxRyZSLSlZSPZwhzffDB3Vr70GDRqEUU1XXx1KhotIZJQsJPNt2wavvBKSxKxZsM8+oVbTb34DTZvGHZ1IjaBkIZlr82YYMSKsI7F4cajdNGRIKMVRt27c0YnUKEoWkpleeAGuuCKsTJeXF1737h1mXotIlVOykMzz9tvws5+FPok33oDu3TWySSRmShaSWQoK4Oyz4YADQv+E+iREMoKShWSOzZtDotiwIdRzUqIQyRhKFpIZ3GHAgNCaePllzZcQyTCqgyCZ4cEH4dln4bbbwvoSIpJRlCwkflOmhLLhvXvDzTfHHY2IlCLSZGFmPcxskZktMbMbS9nfxsymmtl7ZjbXzE5PbG9rZpvMbE7iMSTKOCVGS5dCnz5wyCEwfLiK/olkqMj6LMwsBxgMnAIUALPMbJy7L0w67A/AaHd/3MwOBSYAbRP7lrp7l6jikwywfv32W05jx0LDhvHGIyK7FOXXuK7AEndf5u5bgFFArxLHONAo8bwxsCLCeCSTFBWFZU0XLoTRo+HAA+OOSETKEGWyaA0sT3pdkNiW7DbgPDMrILQqrkja1y5xe+pNM/tBhHFKHP70pzDq6d574eST445GRFKIMlmUNuXWS7zuB/zV3XOB04FnzWwP4DOgjbsfAVwLjDSzRiXOxcwGmFm+meWvWrWqksOXyIwdC7feCuefHyrGikjGizJZFADJdaNz2fk200XAaAB3nwHUBZq7+zfuvjqxfTawFOhQ8gPcfai757l7XosWLSL4FaTSLVgQljjNy4MnnlAZD5EsEWWymAW0N7N2ZlYb6AuMK3HMJ8APAczsEEKyWGVmLRId5JjZAUB7YFmEsUpVWLs2dGjvtVe4BVWvXtwRiUiaIhsN5e6FZnY5MBHIAYa5+wIzGwTku/s44LfAk2Z2DeEW1S/d3c3seGCQmRUC24Bfu/uaqGKVKlBYCH37hmVQp02D3Ny4IxKRcoi03Ie7TyB0XCdvuyXp+UKgWynnvQi8GGVsUsUGDoRJk2DoUDjuuLijEZFy0gwoid7f/hZGPV16KVx8cdzRiEgFKFlItGbPhl/9Co4/PtR/EpGspGQh0fn889Chvc8+MGYM1KoVd0QiUkEqUS7R2LIFzjkHVq+Gf/0LNLRZJKspWUg0rrwyJInnn4cjjog7GhHZTboNJZVvyJAw4e6GG8JwWRHJekoWUrmmT4crroCePeGOO+KORkQqiZKFVJ5PPgn9FAccACNHQk5O3BGJSCVRspDKsXFjWOnum29CocAmTeKOSEQqkTq4Zfe5h8l2770H48bBwQfHHZGIVDIlC9l9994bbjvdcQf86EdxRyMiEUj7NpSZfd/MLkg8b2Fm7aILS7LGa6/BjTfCT38a6j+JSLWUVrIws1uBG4Diq0Et4LmogpIssXhxGBrbqRM884zWphCpxtJtWfQGzgQ2ALj7CqBhVEFJFvjqK+jVC/bcE155JaxRISLVVrp9FlsS60w4gJnpylCTFRWFJVE/+ggmT4Z2uiMpUt2l27IYbWZPAE3M7GJgCvBkdGFJRrvttjDq6YEH4MQT445GRKpAWsnC3e8FxhAWJOoI3OLuj6Q6z8x6mNkiM1tiZjeWsr+NmU01s/fMbK6ZnZ60b2DivEVmdlr6v5JE6sUX4fbb4YIL4PLL445GRKpIyttQibWwJ7r7ycDkdN84cd5g4BSgAJhlZuMSq+MV+wMw2t0fN7NDCavqtU087wscBrQCpphZB3fflu7nSwTmzYP+/eGYY+Dxx9WhLVKDpGxZJC7QG82scTnfuyuwxN2XufsWYBTQq+TbA40SzxsDKxLPewGj3P0bd/8PsCTxfhKX1atDh3ajRqF1UadO3BGJSBVKt4N7MzDPzCaTGBEF4O5XlnFOa2B50usC4OgSx9wGTDKzK4C9gJOTzp1Z4tzWJT/AzAYAAwDatGmTzu8hFVFYCH36wKefwltvQatWcUckIlUs3WQxPvEoj9LuUXiJ1/2Av7r7fWZ2LPCsmXVK81zcfSgwFCAvL2+n/VJJrrsO3ngDhg2Do0vmexGpCdJKFu4+3MxqAx0Smxa5+9YUpxUA+yW9zmX7baZiFwE9Ep8xw8zqAs3TPFeqwvDhYe3sK68MndoiUiOlO4O7O7CY0GH9GPCRmR2f4rRZQHsza5dINH2BcSWO+QT4YeIzDgHqAqsSx/U1szqJsiLtgXfS+o2k8rzzDlxyCZx0Uqj/JCI1Vrq3oe4DTnX3RQBm1gF4Hvjerk5w90IzuxyYCOQAw9x9gZkNAvLdfRzwW+BJM7uGcJvpl+7uwAIzGw0sBAqByzQSqop99lkoOd6yJfz971CrVtwRiUiMLFybUxxkNtfdO6faFqe8vDzPz8+PO4zq4ZtvoHt3mDsXZsyAzhnzv1lEKpmZzXb3vFTHpduyyDezp4FnE6/PBWZXNDjJYO5w2WUwcya88IIShYgA6SeL3wCXAVcSRiq9Rei7kOpm8GB4+mn4wx/gJz+JOxoRyRDpJos9gYfc/X74dna2ZmVVN9OmwdVXw49/DH/8Y9zRiEgGSbeQ4OtAvaTX9QjFBKW6+PjjsIBR+/bw3HOwh5ZnF5Ht0r0i1HX39cUvEs/rRxOSVLkNG+Css2DrVhg7NpT0EBFJkm6y2GBmRxa/MLM8YFM0IUmVcocLLwwjn0aNgg4dUp8jIjVOun0WVwEvmNkKwnyIVsDPIotKqs5dd8Ho0XD33dCjR9zRiEiGSjdZtAOOANoQllg9hlJqNUmWGT8ebroJ+vUL9Z9ERHYh3dtQN7v7V0ATwvoUQ4HHI4tKovfhh/Dzn0OXLvDUU1qbQkTKlG6yKC61cQYwxN3HArWjCUki9+WXoUO7Th145RWor7EKIlK2dG9DfZpYg/tk4G4zq0P6iUYyybZtcO65sHQpvP46aB0QEUlDuhf8PoSCgD3cfR2wN6Cb3Nno5ptDX8XDD8PxqQoHi4gE6a5nsRF4Ken1Z8BnUQUlEfn73+HPf4aLL4Zf/zruaEQki+hWUk0xZ05YvKhbN3j0UXVoi0i5KFnUBKtWhQ7tvfeGMWOgtsYmiEj5RJoszKyHmS0ysyVmdmMp+x8wszmJx0dmti5p37akfSVX2JN0bd0aaj59/nkY+bTvvnFHJCJZKN3RUOWWqEw7mDAvowCYZWbj3H1h8THufk3S8VcQJv4V2+TuXaKKr8a49lp480149lnIS7m+iYhIqaJsWXQFlrj7MnffAowCepVxfD/CUq1SWZ5+OvRP/Pa3cN55cUcj1UD37uEhNU+UyaI1sDzpdUFi207MbH9CSZE3kjbXNbN8M5tpZmdFF2Y1NWMG/OY3cMopof6TiMhuiOw2FGFFvZJ2VU+qLzDG3bclbWvj7ivM7ADgDTOb5+5Ld/gAswHAAIA2mly23aefwtlnw377hUqye0b5v1myzdat8PXXsH799p/Jz8v6+f77kJMDv/gFNG0KTZqER/Hzkj8bNtTAu+oiyqtIAbBf0utcYMUuju1LWLb1W+6+IvFzmZlNI/RnLC1xzFBCnSry8vJU2BBg82bo3Tv8654yJYyAykLFtzqmTYszivi5h+VGynNBT/Vzy5b0P79Bg3DBL/5ZVASFhTB9OqxdGyrHlGWPPXadUMpKMsXP62TJepw14e81ymQxC2hvZu2ATwkJ4eclDzKzjkBTYEbStqbARnf/xsyaA92Av0QYa/XgHibbzZoFL78Mhx0Wd0Q1Tmnf2nfn54YN4X9rOmrX3vHCXvyzZcvSt6f6Wa/ezgsmlrwobtsGX30F69aF5LFu3Y7PS/5ctw5WrNi+bfPmsn+nunXTSzKlbWvcWAs+VqbIkoW7F5rZ5YQyITnAMHdfYGaDgHx3Lx4O2w8Y5b7DP4lDgCfMrIjQr3JX8igq2YWHHoLhw+G228K8CilT8rf2yrrAp/ut3SxclEu7sHfokN4FveTzOKbP5OSEi3PTptCuXfnP37x5exJJlWTWrg0jwD/8cPu2oqJdv7dZWPQxnRZMaT/r1dMttGTm6X5tyXB5eXmen58fdxjxmTIlLF704x/Diy9m/Veq0pr1W7ZUzgW9It/a69Qp3zfzinxrl/IpKgr/L9NJMqVt27Ch7PevXTv9lsztt0OtWqFBv88+oUVUVXb3FpiZzXb3lOPq1fNZHSxbBj/7GRx8MIwYkfVXoc2bw6+0ejW0bVvxb+0lL9KtW6d3IS/tZ61akf7KUgF77BFaDo0aVax48tatOyeUshLP2rXh77L4eWHhzu+5//7hZ6NG8J3v7PjYZ5/SXzdokB0tGCWLbLd+PfTqFb4ijx0brm5ZbPbsMNJm+fLwje2EE8p/Ya9fPzv+8Um8atWCFi3Co7zcYePG7Ynk/PND8rn66nCr7PPPYeXK8HPhQpg6FdasKf296tXbdUIpua1p0/j+tpUssllREfzyl+Gv8bXX4MAD446owrZuhTvvhD/9KfzjOPzwMJBr+PC4IxPZmRnstVd45OaGznSAX/1q1+ds3RrKtBUnk+SEUvz45JMwPmXVqjB4oKQ999w5oSxbVjXrlylZZLM77gj9E/fdFybfZamFC6F/f8jPD+syPfJIGP0rUp3UqgWtWoVHKkVF4TZsyWRSMsEsWAAFBVVzQ0HJIluNHQu33BLKeFxzTerjM1BRETz4IPz+9+H20QsvwE9+EndUIvHbY4/tt8hSjYA/4YSyR4VVFiWLbLRwYUgSeXkwdGhW3qBftiwsr/HWW2EA19ChOxbErc6Tm6T6ifPv1SwMYY5adg+bqYnWrg0d2nvtFcbp1asXd0Tl4h4SQ+fOYT2mZ54JjSRVThfJbGpZZJNt26BvX/jvf8PwitzcuCMqlxUrQgfgq6/CSSeFRKGSXiLZQckimwwcCJMmha/m3brFHU3a3EM9w8suC3MoHnkELr0066eDiGSEqroFpn+u2WLkSLjnnnCVvfjiuKNJ2xdfhPmCP/85dOwYbj1dfrkShUi20T/ZbDB7Nlx0ERx/fBg+lCX+8Q/o1Cms5nrnnaFSaYcOcUclIhWh21CZ7vPPQ1HAFi3C2NIsqDvx1VdhJuszz4SO7IkT4bvfjTsqEdkdShaZbMuWMPFg9Wr45z/D1M0MN3VqmFReUBC6WG69NXvWJBCRXdNtqEx21VUhSTz9NBx5ZNzRlGnjxhDuSSeF5PCvf4VbT0oUItWDWhaZ6oknYMgQuOEG6Ncv7mjK9O9/h+J/H30EV1wRlvyuilo1IlJ11LLIRNOnhyFDPXuG+k8ZassWuOkmOO442LQpLKnx8MNKFCLVUaTJwsx6mNkiM1tiZjeWsv8BM5uTeHxkZuuS9vU3s8WJR/8o48woy5eHfop27cJw2aqYx18Bc+dC167hVtMvfgHz5sEPfxh3VCISlchuQ5lZDjAYOAUoAGaZ2bjk5VHd/Zqk468Ajkg83xu4FcgDHJidOHdtVPFmhE2bwsinTZvCTJsmTeKOaCfbtoXpHrfcEmrrjx0LZ54Zd1QiErUoWxZdgSXuvszdtwCjgF5lHN8PeD7x/DRgsruvSSSIyUCPCGONn3uYbPfee/C3v8Ehh8Qd0U4WL4Yf/CCMcurVK5RHVqIQqRmiTBatgeVJrwsS23ZiZvsD7YA3yntutXHffSFJ3H57KMOaQYqKYPBg6NIFPvgghDl6NDRvHndkIlJVohxmWlaWAAAOhklEQVQNVVrdbN/FsX2BMe5evDZUWuea2QBgAECbbK5IN3FiGPX0k5+ExR0yyPLlcOGFofP6tNPCKN7W1Ttti0gpomxZFAD7Jb3OBVbs4ti+bL8Flfa57j7U3fPcPa9FRRbSzQRLloRKsp06hSnPGbI2hTuMGBGWN50xI4ziffVVJQqRmirKZDELaG9m7cysNiEhjCt5kJl1BJoCM5I2TwRONbOmZtYUODWxrXr56qtw0z8nJxRQatAg7oiAsGzj2WeHpU4PPxzefx8uuSRj8piIxCCy21DuXmhmlxMu8jnAMHdfYGaDgHx3L04c/YBR7u5J564xs9sJCQdgkLuviSrWWBQVwfnnh5lskyaFobIZ4KWXQmL46qsw6umaazJ29K6IVCFLukZntby8PM/Pz487jPTdeisMGgQPPQRXXhl3NKxdG8J47rlQWWTEiNRr/4pI9jOz2e6el+o4zeCOw0svhURxwQWhPkbMJk0Kt5uefz7ksJkzlShEZEdKFlVt3rww5fnoo+Hxx2PtCFi/PqyldNpp0KhRSBK33ZYVVdBFpIopWVSl1avDbLZGjULrIsaSrP/8Z5g3MWQIXHttWF8pL2VDVERqKiWLqlJYGNYX/fRTePllaNUqljA2b4brrw+L7m3bFqqK3Hcf1KsXSzgikiVUoryqXH89vP46DBsWbkHF4N13wx2wBQtgwAC4915o2DCWUEQky6hlURVGjIAHHgjDjS64oMo/vrAwVBE5+mhYswYmTAjLZShRiEi61LKI2jvvhK/xJ54YvspXsQ8+CJPrZs0Kayg9+ijsvXeVhyEiWU4tiyh99hn07g0tW4bKe1U4zKioKDRmjjwSli0LHz9ypBKFiFSMWhZR+eYbOOccWLcuFFeqwhKt//lPuNv15pvwox/Bk0/CvvtW2ceLSDWklkUU3OGyy0KS+OtfoXPnKvvYp54KH/fuu6FC7LhxShQisvvUsojCY4+FK/VNN8FPf1olH/nZZ2HtpPHjQ/fIM8/A/vtXyUeLSA2glkVlmzYNrr463P8ZNKhKPnLUqFCe4/XXQ6mpKVOUKESkcilZVKb//je0JA46KFTk2yPa/7yrV4d5fv36QYcOMGdOGJ0b8ceKSA2ky0pl2bABzjoLtm6FsWOhceNIP278+LBe0ssvwx13hPIdHTtG+pEiUoOpz6IyuMNFF4VVgsaPD1/zI/LVV6GW09NPh0qxr74aajyJiERJLYvKcPfd8Pe/w5//DD17RvYx06aFkU7PPAM33hgm2ilRiEhViDRZmFkPM1tkZkvM7MZdHNPHzBaa2QIzG5m0fZuZzUk8dlqONWNMmAC//31YR/v66yP5iE2bQp/5iSeGeX3Tp4e8FGPRWhGpYSK7DWVmOcBg4BSgAJhlZuPcfWHSMe2BgUA3d19rZvskvcUmd8/s782LFoXe5S5dwn2hCNameOedUPxv0aIwdePuu2GvvSr9Y0REyhRly6IrsMTdl7n7FmAU0KvEMRcDg919LYC7r4wwnsr15ZdhbYo6deCVV6B+/Up9+y1b4Oab4bjjQt/55MmhrpMShYjEIcpk0RpYnvS6ILEtWQegg5n9y8xmmlmPpH11zSw/sf2sCOMsv23b4NxzYelSGDMG2rSp1LefPz9UiP3Tn+C888LieiefXKkfISJSLlGOhirtnoyX8vntge5ALjDdzDq5+zqgjbuvMLMDgDfMbJ67L93hA8wGAAMA2lTyBbtMt9wSRj0NHhxWEaok27aFhYhuvhmaNAnDYs/KrDQpIjVUlC2LAmC/pNe5wIpSjhnr7lvd/T/AIkLywN1XJH4uA6YBR5T8AHcf6u557p7XokWLyv8NSjN6NNx5Z6it8ZvfVNrbLlkS8s4NN4TJ3/PnK1GISOaIMlnMAtqbWTszqw30BUqOanoFOBHAzJoTbkstM7OmZlYnaXs3YCFxe//9UM71uONCB0IldGi7h1JS3/1uWMHu2WfDna2qyn0iIumI7DaUuxea2eXARCAHGObuC8xsEJDv7uMS+041s4XANuA6d19tZscBT5hZESGh3ZU8iioWX3wROrSbNoUXX4TatXf7LQsK4MILQ+f1qaeGAVW5uZUQq4hIJTP3kt0I2SkvL8/z8/OjefOtW8PVfMaMUFcjL2+33s49lI664orw1vfdB5dcEsnIWxGRMpnZbHdPeVFTuY90/Pa3Yfr0iBG7nShWroRf/zp0XnfrBsOHw4EHVk6YIiJRUbmPVIYNg0ceCQWZzj9/t97q5ZdD8b/x4+Evfwkr2SlRiEg2UMuiLDNmhBFPp5wSpk5X0Lp1oXT4s8/CEUfAG2+EpCEiki3UstiVTz+Fs88OPc6jRsGeFcurkyeH6rAjR4b5EzNnKlGISPZRy6I0mzeHRPH11+Fqv/fe5X6LDRtCXcHHHoODDw6NlKOOiiBWEZEqoGRRknvogX7nHXjppQo1A95+G/r3D9VArrkmLE5Ur14EsYqIVBHdhirp4YfDEKVbb4Xevct16jffhBnYP/gBFBbC1Klw//1KFCKS/dSySPb662GY7FlnhfpP5fDee6GU+Pz58KtfhSTRsGFEcYqIVDG1LIotWwZ9+oQOhhEjYI/0/tMUFobqsF27hkne//d/8OSTShQiUr2oZQHhvtF774USHmPHpn2l//DD0DfxzjthobxHH4VmzSKOVUQkBmpZuIer/oYNYR3tNGbJFRXBQw+FORNLloTTnn9eiUJEqi+1LD76CNauhQMOCJPvUvj441B4dto0OOOMcMupZcvIoxQRiZWSRceOYQJEiiqy7qHyxzXXhOdPPx2Shor/iUhNoGQBYcZcGT77DAYMCJ3X3bvDM89A27ZVEpmISEZQn0UKo0eHeXlTpsCDD4bRtUoUIlLTKFnswurV0K8f/OxncNBBYbDUVVelPaJWRKRaifTSZ2Y9zGyRmS0xsxt3cUwfM1toZgvMbGTS9v5mtjjx6B9lnCVNmBBaE2PGwO23w7/+FaZfiIjUVJH1WZhZDjAYOAUoAGaZ2bjk5VHNrD0wEOjm7mvNbJ/E9r2BW4E8wIHZiXPXRhUvhLqB114LTz0VksWECWF4rIhITRdly6IrsMTdl7n7FmAU0KvEMRcDg4uTgLuvTGw/DZjs7msS+yYDPSKMlTffhM6dw4inG26A/HwlChGRYlGOhmoNLE96XQAcXeKYDgBm9i8gB7jN3V/bxbmtS36AmQ0ABgC0adOmQkFu2gTt24flKw48EKZPh+OOq9BbiYhUW1Emi9JmIHgpn98e6A7kAtPNrFOa5+LuQ4GhAHl5eTvtT8fKlfC//0GrVvD++7DXXhV5FxGR6i3KZFEA7Jf0OhdYUcoxM919K/AfM1tESB4FhASSfO60KILcf/9QBLB2bSUKEZFdibLPYhbQ3szamVltoC8wrsQxrwAnAphZc8JtqWXAROBUM2tqZk2BUxPbIvH226F8h4iIlC6yloW7F5rZ5YSLfA4wzN0XmNkgIN/dx7E9KSwEtgHXuftqADO7nZBwAAa5+5qoYhURkbKZe4Vu9WecvLw8z8/PjzsMEZGsYmaz3T0v1XGajywiIikpWYiISEpKFiIikpKShYiIpKRkISIiKSlZiIhISkoWIiKSUrWZZ2Fmq4D/VvHHNge+qOLP3B2KN1qKN1qKNxr7u3uLVAdVm2QRBzPLT2cyS6ZQvNFSvNFSvPHSbSgREUlJyUJERFJSstg9Q+MOoJwUb7QUb7QUb4zUZyEiIimpZSEiIikpWewmM7vHzD40s7lm9rKZNYk7prKY2U/NbIGZFZlZxo7UMLMeZrbIzJaY2Y1xx1MWMxtmZivNbH7csaTDzPYzs6lm9kHib+GquGMqi5nVNbN3zOz9RLx/jDumdJhZjpm9Z2b/F3cslUHJYvdNBjq5e2fgI2BgzPGkMh84G3gr7kB2xcxygMFAT+BQoJ+ZHRpvVGX6K9Aj7iDKoRD4rbsfAhwDXJbh/32/AU5y9+8CXYAeZnZMzDGl4yrgg7iDqCxKFrvJ3Se5e2Hi5UzCeuEZy90/cPdFcceRQldgibsvc/ctwCigV8wx7ZK7vwVkzUqO7v6Zu7+beP414YLWOt6ods2D9YmXtRKPjO5sNbNc4AzgqbhjqSxKFpXrQuDVuIOoBloDy5NeF5DBF7NsZmZtgSOAf8cbSdkSt3TmACuBye6e0fECDwLXA0VxB1JZIluDuzoxsynAvqXsusndxyaOuYnQvP9bVcZWmnTizXBWyraM/iaZjcysAfAicLW7fxV3PGVx921Al0Sf4Mtm1sndM7KPyMx+BKx099lm1j3ueCqLkkUa3P3ksvabWX/gR8APPQPGIqeKNwsUAPslvc4FVsQUS7VkZrUIieJv7v5S3PGky93Xmdk0Qh9RRiYLoBtwppmdDtQFGpnZc+5+Xsxx7RbdhtpNZtYDuAE40903xh1PNTELaG9m7cysNtAXGBdzTNWGmRnwNPCBu98fdzypmFmL4lGGZlYPOBn4MN6ods3dB7p7rru3JfztvpHtiQKULCrDo0BDYLKZzTGzIXEHVBYz621mBcCxwHgzmxh3TCUlBgxcDkwkdL6OdvcF8Ua1a2b2PDAD6GhmBWZ2UdwxpdANOB84KfE3OyfxLThTtQSmmtlcwheJye5eLYajZhPN4BYRkZTUshARkZSULEREJCUlCxERSUnJQkREUlKyEBGRlJQsRCJkZvua2SgzW2pmC81sgpl1iDsukfJSshCJSGLy28vANHc/0N0PBX4PfCfeyETKT+U+RKJzIrDV3b+dqOnuc2KMR6TC1LIQiU4nYHbcQYhUBiULERFJSclCJDoLgO/FHYRIZVCyEInOG0AdM7u4eIOZHWVmJ8QYk0iFqJCgSITMrBVh1bTvAZuBjwmLDS2OMy6R8lKyEBGRlHQbSkREUlKyEBGRlJQsREQkJSULERFJSclCRERSUrIQEZGUlCxERCQlJQsREUnp/wHeKDtQ5u2qhAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.errorbar(x=np.log(param[\"C\"]),y=clf.cv_results_['mean_train_score'],yerr=clf.cv_results_['std_train_score'],label='train',color=\"red\")\n",
    "plt.errorbar(x=np.log(param[\"C\"]),y=clf.cv_results_['mean_test_score'],yerr=clf.cv_results_['std_test_score'],label='test',color=\"blue\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"C\")\n",
    "plt.ylabel(\"score\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "C越大，相当于惩罚松弛变量，希望松弛变量接近0，即对误分类的惩罚增大，趋向于对训练集全分对的情况，这样对训练集测试时准确率很高，但泛化能力弱。C值小，对误分类的惩罚减小，允许容错，将他们当成噪声点，泛化能力较强。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由图可知，当C值很小时，模型过于简单出现欠拟合，随着C值增大，正则效果变小，模型更加复杂，效果提升，到达最优（相对）情况（C=10），之后由于模型复杂度过大开始出现过拟合。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 根据6中得到的训练数据，训练RBF核SVM，并对超参数（C和gamma）进行超参数调优，并分析参数C和gamma对模型复杂度的影响，以及对训练误差和交叉验证得到的测试误差的影响。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 8 candidates, totalling 40 fits\n",
      "[CV] C=1, gamma=1e-05 ................................................\n",
      "[CV] ....... C=1, gamma=1e-05, score=0.5501747378931603, total=   9.4s\n",
      "[CV] C=1, gamma=1e-05 ................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:   15.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... C=1, gamma=1e-05, score=0.5614385614385614, total=   9.2s\n",
      "[CV] C=1, gamma=1e-05 ................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:   30.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... C=1, gamma=1e-05, score=0.5542228885557221, total=   9.3s\n",
      "[CV] C=1, gamma=1e-05 ................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:   45.1s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... C=1, gamma=1e-05, score=0.5655655655655656, total=   9.2s\n",
      "[CV] C=1, gamma=1e-05 ................................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:  1.0min remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ....... C=1, gamma=1e-05, score=0.5586172344689379, total=   9.2s\n",
      "[CV] C=1, gamma=0.0001 ...............................................\n",
      "[CV] ...... C=1, gamma=0.0001, score=0.6989515726410385, total=   6.0s\n",
      "[CV] C=1, gamma=0.0001 ...............................................\n",
      "[CV] ...... C=1, gamma=0.0001, score=0.7057942057942058, total=   6.0s\n",
      "[CV] C=1, gamma=0.0001 ...............................................\n",
      "[CV] ...... C=1, gamma=0.0001, score=0.6951524237881059, total=   6.0s\n",
      "[CV] C=1, gamma=0.0001 ...............................................\n",
      "[CV] ...... C=1, gamma=0.0001, score=0.7077077077077077, total=   6.2s\n",
      "[CV] C=1, gamma=0.0001 ...............................................\n",
      "[CV] ...... C=1, gamma=0.0001, score=0.6978957915831663, total=   6.0s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ........ C=1, gamma=0.001, score=0.746380429355966, total=   4.4s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ....... C=1, gamma=0.001, score=0.7467532467532467, total=   4.4s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ....... C=1, gamma=0.001, score=0.7391304347826086, total=   4.4s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ....... C=1, gamma=0.001, score=0.7527527527527528, total=   4.4s\n",
      "[CV] C=1, gamma=0.001 ................................................\n",
      "[CV] ....... C=1, gamma=0.001, score=0.7354709418837675, total=   4.4s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] ........ C=1, gamma=0.01, score=0.7433849226160759, total=   7.9s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] ........ C=1, gamma=0.01, score=0.7377622377622378, total=   7.8s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] ........ C=1, gamma=0.01, score=0.7321339330334833, total=   7.9s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] ........ C=1, gamma=0.01, score=0.7312312312312312, total=   7.8s\n",
      "[CV] C=1, gamma=0.01 .................................................\n",
      "[CV] ........ C=1, gamma=0.01, score=0.7279559118236473, total=   7.8s\n",
      "[CV] C=10, gamma=1e-05 ...............................................\n",
      "[CV] ...... C=10, gamma=1e-05, score=0.6994508237643535, total=   6.0s\n",
      "[CV] C=10, gamma=1e-05 ...............................................\n",
      "[CV] ...... C=10, gamma=1e-05, score=0.7047952047952047, total=   6.0s\n",
      "[CV] C=10, gamma=1e-05 ...............................................\n",
      "[CV] ...... C=10, gamma=1e-05, score=0.6951524237881059, total=   5.9s\n",
      "[CV] C=10, gamma=1e-05 ...............................................\n",
      "[CV] ...... C=10, gamma=1e-05, score=0.7072072072072072, total=   6.0s\n",
      "[CV] C=10, gamma=1e-05 ...............................................\n",
      "[CV] ...... C=10, gamma=1e-05, score=0.6958917835671342, total=   6.0s\n",
      "[CV] C=10, gamma=0.0001 ..............................................\n",
      "[CV] ..... C=10, gamma=0.0001, score=0.7398901647528707, total=   4.0s\n",
      "[CV] C=10, gamma=0.0001 ..............................................\n",
      "[CV] ..... C=10, gamma=0.0001, score=0.7372627372627373, total=   4.0s\n",
      "[CV] C=10, gamma=0.0001 ..............................................\n",
      "[CV] ..... C=10, gamma=0.0001, score=0.7306346826586707, total=   4.0s\n",
      "[CV] C=10, gamma=0.0001 ..............................................\n",
      "[CV] ..... C=10, gamma=0.0001, score=0.7442442442442443, total=   4.1s\n",
      "[CV] C=10, gamma=0.0001 ..............................................\n",
      "[CV] ..... C=10, gamma=0.0001, score=0.7359719438877755, total=   4.0s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ...... C=10, gamma=0.001, score=0.7753369945082377, total=   3.7s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ...... C=10, gamma=0.001, score=0.7807192807192808, total=   3.7s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ...... C=10, gamma=0.001, score=0.7611194402798601, total=   3.6s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ...... C=10, gamma=0.001, score=0.7662662662662663, total=   3.6s\n",
      "[CV] C=10, gamma=0.001 ...............................................\n",
      "[CV] ...... C=10, gamma=0.001, score=0.7605210420841684, total=   3.8s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ....... C=10, gamma=0.01, score=0.7533699450823764, total=   8.4s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ....... C=10, gamma=0.01, score=0.7367632367632367, total=   8.4s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ....... C=10, gamma=0.01, score=0.7476261869065467, total=   8.3s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ....... C=10, gamma=0.01, score=0.7467467467467468, total=   8.1s\n",
      "[CV] C=10, gamma=0.01 ................................................\n",
      "[CV] ....... C=10, gamma=0.01, score=0.7329659318637275, total=   7.9s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  40 out of  40 | elapsed:  7.1min finished\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "svc2 = SVC(kernel='rbf')\n",
    "param2 = {\"C\":[1,10],\"gamma\":[1e-5,1e-4,1e-3,1e-2]}#为节省时间，C选择范围根据经验减少\n",
    "clf2 = GridSearchCV(svc2,param2,return_train_score=True,cv=5,verbose=5)\n",
    "clf2 = clf2.fit(x_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "最佳训练效果得分： 0.7688\n",
      "最佳超参数 {'C': 10, 'gamma': 0.001}\n"
     ]
    }
   ],
   "source": [
    "print(\"最佳训练效果得分：\",clf2.best_score_)\n",
    "print(\"最佳超参数\",clf2.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([7.95974131, 5.05653272, 3.57464023, 6.76235967, 4.99224458,\n",
       "        3.25117531, 2.92824807, 7.11813703]),\n",
       " 'std_fit_time': array([0.06675639, 0.08104151, 0.0163448 , 0.04713718, 0.01192086,\n",
       "        0.02042433, 0.09155217, 0.14781218]),\n",
       " 'mean_score_time': array([1.41657476, 1.08165836, 0.93149648, 1.16867514, 1.06542625,\n",
       "        0.888237  , 0.86166105, 1.21395421]),\n",
       " 'std_score_time': array([0.01219343, 0.02237478, 0.00400207, 0.01156499, 0.00490498,\n",
       "        0.0122597 , 0.01188452, 0.03991499]),\n",
       " 'param_C': masked_array(data=[1, 1, 1, 1, 10, 10, 10, 10],\n",
       "              mask=[False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_gamma': masked_array(data=[1e-05, 0.0001, 0.001, 0.01, 1e-05, 0.0001, 0.001, 0.01],\n",
       "              mask=[False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'C': 1, 'gamma': 1e-05},\n",
       "  {'C': 1, 'gamma': 0.0001},\n",
       "  {'C': 1, 'gamma': 0.001},\n",
       "  {'C': 1, 'gamma': 0.01},\n",
       "  {'C': 10, 'gamma': 1e-05},\n",
       "  {'C': 10, 'gamma': 0.0001},\n",
       "  {'C': 10, 'gamma': 0.001},\n",
       "  {'C': 10, 'gamma': 0.01}],\n",
       " 'split0_test_score': array([0.55017474, 0.69895157, 0.74638043, 0.74338492, 0.69945082,\n",
       "        0.73989016, 0.77533699, 0.75336995]),\n",
       " 'split1_test_score': array([0.56143856, 0.70579421, 0.74675325, 0.73776224, 0.7047952 ,\n",
       "        0.73726274, 0.78071928, 0.73676324]),\n",
       " 'split2_test_score': array([0.55422289, 0.69515242, 0.73913043, 0.73213393, 0.69515242,\n",
       "        0.73063468, 0.76111944, 0.74762619]),\n",
       " 'split3_test_score': array([0.56556557, 0.70770771, 0.75275275, 0.73123123, 0.70720721,\n",
       "        0.74424424, 0.76626627, 0.74674675]),\n",
       " 'split4_test_score': array([0.55861723, 0.69789579, 0.73547094, 0.72795591, 0.69589178,\n",
       "        0.73597194, 0.76052104, 0.73296593]),\n",
       " 'mean_test_score': array([0.558 , 0.7011, 0.7441, 0.7345, 0.7005, 0.7376, 0.7688, 0.7435]),\n",
       " 'std_test_score': array([0.0053851 , 0.00481531, 0.00610034, 0.00545395, 0.00478153,\n",
       "        0.00448673, 0.00798556, 0.00750068]),\n",
       " 'rank_test_score': array([8, 6, 2, 5, 7, 4, 1, 3]),\n",
       " 'split0_train_score': array([0.55895961, 0.70438915, 0.76016006, 0.85369514, 0.70338877,\n",
       "        0.75153182, 0.82218332, 0.96336126]),\n",
       " 'split1_train_score': array([0.56264066, 0.70380095, 0.76019005, 0.8572143 , 0.70305076,\n",
       "        0.75331333, 0.82645661, 0.96436609]),\n",
       " 'split2_train_score': array([0.55994499, 0.70596325, 0.76384548, 0.85860733, 0.70446306,\n",
       "        0.75234404, 0.82485311, 0.96287036]),\n",
       " 'split3_train_score': array([0.5576106 , 0.70407398, 0.76455886, 0.86128468, 0.70294926,\n",
       "        0.75431142, 0.82704324, 0.9643839 ]),\n",
       " 'split4_train_score': array([0.56209395, 0.70414793, 0.76574213, 0.85569715, 0.70327336,\n",
       "        0.7529985 , 0.82321339, 0.9623938 ]),\n",
       " 'mean_train_score': array([0.56024996, 0.70447505, 0.76289932, 0.85729972, 0.70342504,\n",
       "        0.75289982, 0.82474993, 0.96347508]),\n",
       " 'std_train_score': array([0.00188892, 0.00076738, 0.00230541, 0.0025752 , 0.00054189,\n",
       "        0.00093313, 0.00185101, 0.00079595])}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf2.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'score')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl4FFW6+PHvyUI2wr4TlrAIhC1gQCChwzjixggKF0UQcQHUnwvgDDO4L3e2e4d7ddwHR3G86nUcFXXGbeSOEFYhbAIB2QkJCCFA9q3T5/dHdSrpkJBO0pVKp9/P8+Shl1OVt5Jw3j5Vdc6rtNYIIYQQAEF2ByCEEKL5kKQghBDCJElBCCGESZKCEEIIkyQFIYQQJkkKQgghTJIUhBBCmCQpCCGEMElSEEIIYQqxO4D66tSpk+7bt6/dYQghhF/Ztm3bWa1157ra+V1S6Nu3L6mpqXaHIYQQfkUpddybdnL6SAghhEmSghBCCJMkBSGEECZJCkIIIUySFIQQQpgkKQghhDBJUhBCCGGSpCCEEH7gxIkTnD9/3vLv43eT14QQoqXTWnP48GFSUlJISUlh7dq1HDt2jIEDB3LgwAFLv7ckBSGEsJnL5WLfvn1mAkhJSeHUqVMAdOrUCYfDweLFi7nhhhssj0WSghBCNLHy8nJ27dplJoB169aRnZ0NQM+ePfnJT36Cw+HA4XAwePBglFJNFpskBSGEsFhpaSmpqanm6aANGzaQm5sLQP/+/Zk6daqZBGJjY5s0CVQnSUEIIXyssLCQ7777zkwCmzZtoqioCIC4uDhmz56Nw+Fg4sSJxMTE2BytJ0kKQgjRSLm5uWzcuNG8JrB161bKyspQShEfH8/ChQvNJNC5c52rV9tKkoIQQtRTdnY269evN5PAjh07cLlchISEkJCQwJIlS0hOTmbChAm0a9fO7nDrRZKCEELU4dSpU+apoJSUFPbs2QNAeHg448aN4/HHH8fhcDBu3DiioqJsjrZxJCkIIUQ1x48fN+8MSklJ4eDBgwC0bt2axMREbr31VhwOB2PGjCEsLMzmaH1LkoIQIqBprTlw4IDHSCA9PR2A9u3bM3HiRO655x4cDgejRo0iJKRld5st++iEEKIal8vFnj17PJLA6dOnAejatSsOh4OlS5eSnJzM0KFDCQoKrNWAJCkIIVo0p9PJjh07zIvC69evN9cQ6tWrF5MnTyY5ORmHw8HAgQNtnSPQHEhSEEK0KCUlJWzdutW8JrBx40by8/MBGDhwIDNmzDAnivXp08fmaJsfSQpCCL9WUFDApk2bzFNBmzdvpqSkBIDhw4czb948c45A9+7dbY62+ZOkIITwKxcuXGDDhg1mEkhNTcXpdBIUFMTo0aO5//77cTgcJCUl0bFjR7vD9TuSFIQQzVpWVhbr1q0zrwns2rULrTWhoaGMHTvWvCg8fvx42rRpY3e4fk+SghCiWcnMzPRYQnrfvn0AREREMH78eJ566imSk5O54ooriIiIsDnalkeSghDCNlprjh496jFR7MiRIwBER0eTlJRkXhO4/PLLadWqlc0Rt3ySFIQQTUZrbRaTqfjKzMwEoGPHjjgcDh588EEcDgcjR44kODjY5ogDjyQFIYRlysvL+f777z2SwNmzZwHo3r27OT/A4XAwZMiQgJso1hxJUhBC+ExZWRnbtm3zmChWUUwmNjaWKVOm4HA4SE5Opl+/fgE/Uaw5kqQghGiwoqIitmzZYiaBTZs2UVhYCMDgwYOZNWsWycnJTJw4kV69etkcrfCGJAUhhNfy8vLMYjIpKSls2bKF0tJSlFKMHDmS+fPnmxPFunTpYne4ogEkKQghanX+/HlzjkBKSgrbt2+nvLyc4OBgEhISWLRoEQ6Hg8TERNq3b293uMIHJCkIIUw//vijRxLYvXs3WmvCwsK44ooreOSRR3A4HIwfP57WrVvbHa6wgCQFIQJYenq6mQDWrl3LgQMHAIiKimLChAnMnDmT5ORkxowZQ3h4uM3RiqYgSUGIAKG15tChQx6zhY8fPw5A27ZtmThxIgsWLDCLyYSGhtocsbCDpUlBKXUt8EcgGPiz1vr31d7vA7wJdAbOAbdprTOsjEmIQOFyuUhLS/OYLfzjjz8C0KVLFxwOBz//+c9xOBwMGzZMJooJwMKkoJQKBl4GJgMZwFal1Gda67QqzZYDb2ut/6KUuhL4HTDXqpiEaMmcTic7d+40E8C6des4d+4cADExMfz0pz81J4oNGjRI5giIGlk5UhgLHNJaHwFQSr0PTAOqJoU4YIn78bfAJxbGI0SLUlJSQmpqqpkENmzYQF5eHgADBgzgxhtvNCeK9enTR5KA8IqVSaEncKLK8wzgimptdgEzME4x3QREK6U6aq2zLYxLCL9UWFjI5s2bzWsCmzdvpri4GIChQ4dy2223mSOBHj162Byt8FdWJoWaPpboas9/AbyklLoDSAEyAedFO1JqIbAQoHfv3r6NUohmKicnh40bN5rXBFJTUykrKyMoKIj4+Hjuu+8+s5hMp06d7A5XtBBWJoUMoOq89hjgZNUGWuuTwHQApVRrYIbWOqf6jrTWK4AVAAkJCdUTixAtwtmzZ1m/fr2ZBHbu3InL5SI0NJQxY8aYF4UnTJhA27Zt7Q5XtFBWJoWtwEClVCzGCGAWMLtqA6VUJ+Cc1toFPIJxJ5IQAeHkyZMeq4fu3bsXgPDwcMaPH88TTzyBw+Fg3LhxREZG2hytCBSWJQWttVMp9QDwNcYtqW9qrfcqpZ4FUrXWnwGTgN8ppTTG6aP7rYpHCDtprTl27JhHEjh06BBgFJNJTExkzpw5OBwOEhISCAsLszliEaiU1v51NiYhIUGnpqbaHYYQl6S15ocffvCYLZyRYUzB6dChAxMnTjTvDBo5ciQhITKPVFhLKbVNa51QVzv5SxTCB1wuF7t37/aYLZyVlQVAt27dPIrJxMXFSTEZ0WxJUhCiAcrKyti+fbs5Eli/fj0XLlwAoE+fPlx33XVmEhgwYIDMERB+Q5KCEF4oLi42i8mkpKSwceNGCgoKABg0aBAzZ8406wj06dPH5miFaDhJCkLUID8/n02bNplJ4LvvvqOkpASlFMOHD+fOO+80RwJdu3a1O1whfEaSghAYxWQ2bNhgJoFt27bhdDoJDg5m9OjRPPjgg2YxmQ4dOtgdrhCWkaQgAtKZM2dYt26deVH4+++/R2tNq1atGDt2LL/61a/MYjLR0dF2hytEk5GkIAJCRkaGxxLS+/fvByAyMpIJEybwzDPP4HA4GDt2LBERETZHK4R9JCmIFkdrzeHDhz0mih09ehQwiskkJSWZ1wRGjx5Nq1atbI5YiOZDkoLwey6Xi3379nkkgZMnjWW2OnXqhMPhYNGiRSQnJzN8+HApJiPEJUhSEH6nvLycXbt2eSSB7GxjtfWePXsyadIk886gwYMHyxwBIepBkoJo9kpLS9m2bZt5TWDDhg3k5uYC0K9fP6ZOnWomgdjYWEkCQjSCJAXR7BQVFfHdd9+ZSWDTpk0UFRUBEBcXx+zZs82JYjExMTZHK0TLIklB2C43N5eNGzeap4K2bNlCWVkZSini4+NZuHChmQQ6d+5sd7hCtGiSFESTy87OZv369WYS2L59Oy6Xi5CQEBISEliyZAnJyclMmDCBdu3a2R2uEAFFkoKw3KlTp1i3bp25guiePXsACAsLY9y4cTz++ONmMZmoqCiboxWiGXK5YPx4CAmBDRss/VaSFITPHT9+3GMJ6YMHDwLQunVrEhMTufXWW3E4HIwZM0aKyQhRVXk5HDsGaWmwd6/xb1oa7NsHhYUwcKDlIUhSEI2itebgwYMes4XT09MBaN++PRMnTuSee+7B4XAwatQoKSYjBBid/5EjNXf+xcWV7Xr2hKFDYeFC+OILaILTqVJ5TdSLy+Viz549HnMETp8+DUDXrl3NW0OTk5MZOnSoFJMRgc3phMOHPTv+vXvhhx+gpKSyXa9eRucfF2d8DR0KQ4ZA27Y+C0UqrwmfcDqd7Nixw0wA69at4/z58wD06tWLyZMnm0lg4MCBMkdABKayMjh06OLO/8ABKC2tbNenj9HhX311Zec/eDC0aWNf7NVIUhAeSkpK2Lp1q5kENmzYQH5+PgADBw5k+vTpZmlJKSYjAk5pKRw8WHPn73QabZSC2Fij07/+es/Ov3Vre+P3giSFAFdQUMDmzZvNawKbN2+mxD2sHT58OPPmzTPnCHTv3t3maIVoIiUlRkdfvfM/eNC4HgBG59+/v9HpT51aefpn0CDw47voJCkEmJycHI85AqmpqTidToKCghg9ejT3338/DoeDpKQkOnbsaHe4QliruNg4v1+98z90yLgNFCAoCAYMMDr86dM9O/8WuMy6JIUWLisry5wjkJKSws6dO9FaExoaytixY1m6dCkOh4MJEybQphmd1xTCpwoLYf/+yo6/ovM/cqSy8w8ONm75HDYMbr65svO/7DIID7c3/iYkSaGFyczM9LgzKC0tDYCIiAjGjx/PU089RXJyMldccYUUkxEtT0GBcVtn1Y4/LQ2OHoWKOy1DQoyOPj4eZs+u7PwHDgSZNyNJwZ9prTl69KiZANauXcuRI0cAiI6OJikpidtvvx2Hw8Hll18uxWREy5GXV3Pnf+xYZZvQUOMUT0IC3H57Zec/YADI/4VaSVLwI1pr9u/f7zFbODMzE4COHTvicDjMAvMjR46UYjLC/+XmXnzKJy0N3BMkAaODHzwYxo2Du++uvNe/f38jMYh6kaTQjJWXl7N7924zAaxbt46srCwAunfvbt4a6nA4GDJkiEwUE/7rwoWaO/+MjMo24eFG55+U5DnRq18/45SQ8An5STYjZWVlbNu2zTwdtH79enJycgCIjY3l+uuvN5NA//79ZaKY8D/nzl3c8aelgbt8KmDc0TNkCEya5Nn5x8YaF4OFpSQp2KioqIgtW7aYSWDjxo0UFhYCMHjwYG655RaSk5OZOHEivXr1sjlaIerh7NmaO/8ff6xsExlpdPZXXeXZ+ffta9wGKmwhSaEJ5eXlsWnTJvOawJYtWygtLUUpxciRI5k/f745UaxLly52hytE3c6cqbnzP3Omsk3r1kZnf+21np1/797S+TdDkhQslpSURHp6Ot26dWP79u2Ul5cTHBxMQkICixYtwuFwkJiYSPv27e0OVYiaaQ2nT9fc+Z89W9muTRujs7/hhsqOPy7OWOxNTnX6DUkKFnK5XOzbt49z584RGxvLI488gsPhYPz48bT2gzVQRIDRGk6dqrnzP3eusl3btsYn/ptu8lzVs0cP6fxbAEkKFnr66ac5d+4cr7zyCvfdd5/d4Qhh0BoyMy/u+NPSjLuAKrRvb3T2M2d6dv7duknn34JJUrDIp59+yr//+79z1113ce+999odjghEWsOJEzV3/rm5le06djQ6+1tv9ez8u3SRzj8ASVKwwP79+5k7dy5jxozh5ZdflltHhbVcLmMyV02dv3vZc8Do5OPiYO5cz86/c2f7YhfNjiQFH8vNzeWmm24iPDycjz76iPAAWkhLWMzlMpZxqN7x79tnrPlToVs3o8O/807PC76dOtkWuvAfkhR8yOVyMW/ePA4ePMjq1atlboFomPJyYwG36ss5798PRUWV7Xr0MD7pz5/vWcKxQwf7Yhd+T5KCD/3ud7/jk08+4bnnnmPSpEl2hyOaO6fTWLq5ps6/av3emBijw580ybPzb4Ii7iLwSFLwkS+//JInnniCOXPmsGjRIrvDEc1JWVntxdur1u/t3dvo8K+6yrPzlzoXoglZmhSUUtcCfwSCgT9rrX9f7f3ewF+Adu42y7TWX1gZkxUOHTrE7NmzGTlyJCtWrJALy4GqtLT24u1lZZXt+vY1Ovxrr/Ws3xsdbVvoQlSwLCkopYKBl4HJQAawVSn1mdY6rUqzx4EPtNavKqXigC+AvlbFZIX8/HxuuukmgoKC+Pjjj4mMjLQ7JGG1kpKai7cfPOhZvL1fP6PT/9nPPDt/P67fK1o+K0cKY4FDWusjAEqp94FpQNWkoIGKsXFb4CR+RGvN3XffTVpaGl999RWxsbF2hyR8qaJ+b/UZvocOVRZvDwqqLN5+442e9XvlA4LwQ1YmhZ7AiSrPM4ArqrV5GvinUupBIAq4ysJ4fG758uV88MEH/Md//AeTJ0+2OxzRUEVFnvV7Kzr/w4c96/dWFG//t3/zrN8rZU1FC2JlUqjpxLqu9vxW4C2t9X8ppcYD/6OUGqa1dnnsSKmFwEKA3r17WxJsfa1evZply5Yxc+ZMli5danc4whuFhTWXcDxyxLN+78CBMGIEzJrl2flL/V4RAKxMChlA1Rv1Y7j49NDdwLUAWutNSqlwoBNwpmojrfUKYAVAQkJC9cTS5I4dO8asWbMYMmQIb775plxYbm7y82uv31vR+YeGGh396NFw222exdulfq8IYFYmha3AQKVULJAJzAJmV2uTDvwUeEspNQQIB7IsjKnRioqKmD59Ok6nk08++URWO7VTbm7Nnf/x45VtWrUyzu+PHQt33OFZvF3q9wpxEcuSgtbaqZR6APga43bTN7XWe5VSzwKpWuvPgJ8DryullmCcWrpDa237SKA2WmsWLlzIzp07+fvf/86AAQPsDikw5OTUvJzziSqXrMLCjDt7JkyABQs8i7dL/V4hvGbp/xb3nIMvqr32ZJXHaUCilTH40osvvsg777zDs88+y5QpU+wOp+U5f77mzj8zs7JNeLgxocvhuLh4u9TvFaLR5COUl9auXcvDDz/MtGnTeOyxx+wOx79lZ9fc+Z86VdkmMtLo/K+88uL6vdL5C2EZSQpeyMjI4Oabb2bAgAG8/fbbBEldWe9kZdXc+Z8+XdkmKsro7K++2rPz79NH6vcKYQNJCnUoLi5mxowZFBUVsXbtWtrIOjSetK65ePvevZ71e6Ojjc5+ypSL6/dK5y9EsyFJ4RK01jzwwANs2bKFVatWMXjwYLtDso/W8OOPNRdyyc6ubNe2beXs3qqFXHr2lCpeQvgBr5OCUioJGKi1XqmU6gy01loftS40+61YsYI33niDxx57jBtvvNHucJqG1nDyZM2d//nzle3atTM6+xkzPDv/7t2l8xfCj3mVFJRSTwEJwCBgJRAKvIMf3TlUX5s2beLBBx/kuuuu45lnnrE7HN/TGjIyau78c3Iq23XoYHT2t9zi2fl37SqdvxAtkLcjhZuAUcB2AK31SaVUi13n99SpU8yYMYPevXvz7rvvEtyYu11WTzL+vWqNL0KrP5er9uLteXmV7Tp3Njr8OXMurt8rnb8QAcPbpFCqtdZKKQ2glGqxa/+WlpYyc+ZMcnJy+Prrr2nfvr3dIXnH5TJm8lbv+NPSPOv3du1qdPjz5nle8JXi7UIIvE8KHyil/gS0U0otAO4CXrcuLPssWbKEDRs28Ne//pXhw4fbHc7FyssvLt6+d6+x3EPV+r3duxuf9O++27OKV8eOtoUuhGj+vEoKWuvlSqnJQC7GdYUntdbfWBqZDVauXMkrr7zC0qVLufnmm32z09IcKP4RvlsIIVHur9Z1P1YRcOIMpB2oXN+non5vcXHl/nv2NDr8e+7x7Pz9ZYQjhGhWVF1LDbkrqH2ttW4WtQ4SEhJ0amqqz/ebmppKUlISSUlJfPXVV4Q0dr2c3B9g5yOQsQpUMIR1BmcBlBeA58rgl1YOFAPOYCAMQltDWFto3RHadoGIdt4lmeqPgyMhSGYGCxEolFLbtNYJdbWrs+fTWpcrpQqVUm211jl1tfdHZ86cYfr06XTr1o3333+/cQmh6EfY/Qwcft3oeKP6QmQMTPqXUbFrzx7Y/z0c3AOH0+DkEQhyGuvDhgExnaBvd+Pfbu2hczS0jYDgMijLr0wsZflQdAjyCsBZ8XpxHcFVExxec+IIjjKST22Jper7NT0OCpOL00L4KW97v2Jgt1LqG8C8aqm1fsiSqJqQ0+nklltuISsri40bN9KpU6eG7agsH/b/F+z7A5SXwMD/B8OegKWx8FE6ZEV61u+NjTVO9yTd6Fm/tzFLcbvKjYThrPjKr8fjKq+VZriTTZXXdbn3caigygQR7E4oVR83ZGRT8ThI5lsKYSVv/4d97v5qcX75y1+yZs0a3n77bUaNGlX/HbiccPgN2P0UFJ+G3jNh5G8hegBs3Qp/LoAIBb/4RWXnP2iQNcXbg4IhqA2E+ngpDq3BVVrPJFPD47IcKDrpmYjKi+r+/h7HGNb4xFLj6CdCRjdC4P2F5r8opVoBl7lf+kFrXWZdWE3jvffe47nnnuOhhx5i7ty59dtYa8j4FHYtM64fdE4CxyfQaZzx/o4dxiJv3WNh7VpjjR9/pRQEhxlfYT6+e0m7wFnY+IRTdKrytFrFa9pZn4OskihqSyINTDhBUsxH+A9vZzRPAv4CHMOovdxLKTVPa51iXWjW2rlzJ/Pnz8fhcLB8+fL6bZy1CXYuhawN0GYwOD6FnjdUftL8/nu46ipo0wb+9S//TghWU0HGqaVQCyrYlZdWnk4ry/dMGnU9rrpd8emLE1F9BIV6f22m+um2S127CY4wfn5C+JC3p4/+C7haa/0DgFLqMuB/gcutCsxK2dnZTJ8+nQ4dOvDBBx8Q6m1ZxtwDsOtROPERhHeDsX+Cfnd5nufeuxd++lOIiDASQt++lhyD8EJwK+OrlY9vz9Uu47SXt9doantccgYKjnq+7iqt5zFG+u56TdURUbDUqQ5U3iaF0IqEAKC1PqCU8tsx8bBhwzh9+jSbN2+ma9eudW9QfAZ2PwuH/mTcsTP8GRj88MWfbn/4wUgIISFGQujf35oDEPZSQZUdqa+5yqokkEbcLFCSdfH71KPSrQqx5tpNSJSMbpo5b5NCqlLqDeB/3M/nANusCcl6Xbt25fLLL2fs2LGXbugsgH3/Dfv+0/hkOOAeGPYkRNSQSA4dMqqEaQ3ffguXXXZxGyHqEhQKrdoZX76ktXHLcmOv3ZRkQ8Fxz1NtrpL6xRIcUb9Ri9fXbuRWaF/wNincB9wPPIRxTSEFeMWqoKx24sQJJkyYUHsDlxOOrDTuKCo6Bb2mG3cUtRlUc/ujR42EUFICa9YYM4rdJk0y/l2zxlfRC9EASkFIhPGFj9e5cjmhvND7azQ1PXbmQ+GJi0c+9ZnoqYLqd73Gm1ulK95vDhM9m2hxTW+TQgjwR631f4M5yznMsqgsVFBQwLlz5+jdu/fFb2oNmf+Anb+C3H3QaQIkfQidL5FA0tONhJCfb5wyGjbMuuCFaI6CQiy8Fbqk4aOaisdlF6Ao0/P1htwK7fNrN83zVmhvk8L/AVcB+e7nEcA/gUv0ls3TiRMnAOhV/Y6gs9/Bzl/CmRSIvgwmfgwxN176F5aZCT/5iVF85v/+D+LjL2py7pyRN5KSjEsNISEQGlr3Y7vaVd+mMauGC9EoShnX8ILDfX8rtKvcGN005AaBqo+LTl78eoNvha7j9Fn+Md//HGrgbVII11pXJAS01vlKqUiLYrJUeno6QOVIIe+QcUdR+t8gvAuMeQX6z6/73vJTp4wRQlYWfPMNXO55I1Z5OTzzDOzeDeHhxpfTaSxkmpdnPC4rM/6t/rim91z1GEX7klL+k8Aa2y4kpNl9aBNWCQqGoGgItaAsTHl9J3rW8FpZnnHq2uMUXKExV8hi3iaFAqXUaK31dgClVAJQz/FX81AxUuhzfDFEJMLBVyGoFQx7Cob83Ls/kjNnjLuMMjPh66/hiis83v7xR5g927je3LUrDBwIq1c3Lm6Xy0g0dSUPb5OMr7apq11RUcNisEtwsL2JSUaBLUBwKwjuAGEdfLvfb5J9u79aeJsUFgF/U0qdxLivrQdwi2VRWSg9PZ2ZV0CvoF1wcJcxKhj+FER0924HZ88aE9OOHYMvv4REz4qk//qXkRByc2HlSnjrLd/EHRRkfHk7pcKfaW0kwaZOYI3dt4wCZRRopZ27jAOIn2zt9/E2KcRilOPsjVGacxz1uum5+UhPT+fRGxUquBVcuw3aDql7owrnzsHkyXDwIPzjH5BcmbnLy+E3vzFOGV12mTEyGDYM7rjD98fQ0illfIoNDoYwv7ydoX5qGgVamfR8lRwrRoH13bdd/H0UeOhkP7q2y7L85+RtUnhCa/03pVQ7YDLGDOdXgSsuvVnzc+JEOgMd2pjlWp+EcOGCsZZRWhp8+qlx+sjtzBmjtPHq1XDbbfDqq41b7FQEFhkF2pP06hODN6PAmvbt21HgmwwcCAee9OU+L+ZtUqhYN3kK8JrW+lOl1NPWhGStonNHiQoD4h7xfqPcXLjuOmNNo48/hmuvNd9auxZuvdW4AenPf4a77vL/YaoQVpFRYMMT2COPNM2HTW+TQqa7RvNVwH8opcIAv5urrrWmtSvDeNJ2sHcb5efDlCnGMth/+xv87GeA8cv+3e/gySdhwAD46isYMcKiwIUQfsmXo8D//u/G78Mb3iaFm4FrgeVa6wtKqe7AUuvCssbZs2eJ7eRe8Tu6ltnJVRUWwg03wMaN8P77cNNNgHEX6ty5xo1Ht94Kf/oTRFtwZ5sQQlRoqlURvEoKWutC4OMqz08Bp6wKyionTpxgcHdwEkZIZM9LNy4qgmnTjPND77wDM2cCsG4dzJoF2dlGMliwQE4XCSFaDr87BdQY6enpDOoBpWF9L71SY0kJTJ9uzFJeuRJmz8blgt//3pjAHBkJmzfDwoWSEIQQLUtAJYUTJ04wqDsEtbvEXUelpcao4KuvYMUKmDePs2eNs0iPPAIzZsC2bTWuaCGEEH4voJLCyROH6dsJwjrXckW4rMw4N/T3v8Mrr8D8+WzYAKNGGbebvvKKcWmhjY/X/RJCiOYioJKC8/x+goJAtalhpOB0GlePV62C55/Hdc99/OEPxvy0Vq1g0ya47z45XSSEaNm8vfuoRWhVfNR4UL0uQnk53Hkn/PWv8Ic/cG7uIuZNMyYtz5gBb7wBbds2fbxCCNHUAioptFXuG6baVKmK5nLB/PnGHUa/+Q2bk37BLaOMRVBffBHuv19GB0KIwBEwp4/Kysro0TqPC2VtMGvrulxw773w1lvoJ5/iuYhHmTjRmG25cSN9dCo1AAAWIElEQVQ88IAkBCFEYAmYkcLJkycZ1B3yg3rSDoxFWB56CF5/nfNLnuXOXY/z6afG/LQ334R2Pi6RK4QQ/sDSkYJS6lql1A9KqUNKqWU1vP+cUmqn++uAUuqCVbGccM9RKI8aaCSEhx+Gl19m65znGb3qcb74QvH88/DRR5IQhBCBy7KRgruO88sYq6pmAFuVUp9prdMq2mitl1Rp/yDG8tyWOHPie9pEQEHHYbw6YDn3HnmeFx0f8osPptOjh2L9ehg71qrvLoQQ/sHKkcJY4JDW+ojWuhR4H5h2ifa3Av9rVTCZq43807bXWIacXce1watZlDKD665TbN8uCUEIIcDaawo9gRNVnmdQS/0FpVQfjEI+/7IqmLmTY0AfJbLraJ4rDOX/ypNZvtw4iyQXk4UQwmDlSKGmrra2am2zgA+11uU1vamUWqiUSlVKpWZlNazyULs2kRDWCSJ7csrVheEh+/n5zyUhCCFEVVYmhQygV5XnMcDJWtrO4hKnjrTWK7TWCVrrhM6dOzcsmrBO0HYoqCBKdCtaBZU1bD9CCNGCWXn6aCswUCkVC2RidPyzqzdSSg0C2gObLIwFrlpjPnQGh9G7Z56l304IIfyRZSMFrbUTeAD4GtgHfKC13quUelYpNbVK01uB97XWtZ1a8rmC8ggiw3xaPFUIIVoESyevaa2/AL6o9tqT1Z4/bWUMNSnU4URGSFIQQojqAmaZC1NZGYVEEhXRZAMTIYTwGwGXFHRhEYVEEhlpdyRCCNH8BFxSKDlfiCaIyNZyL6oQQlQXcEmhILsYgMiogDt0IYSoU8Csklqh8HwJAJHRkhSEaCplZWVkZGRQXFxsdygtXnh4ODExMYSGhjZo+8BLCueMP8qo6GCbIxEicGRkZBAdHU3fvn1RsoyAZbTWZGdnk5GRQWxsbIP2EXAflwsvlAIQ2Sbg8qEQtikuLqZjx46SECymlKJjx46NGpEFXlLIMZa3iGzbsKGVEKJhJCE0jcb+nAMuKRTkOAGIbNfK5kiEEJc0aZLxJZpUwCWFwlwjKUR1CLM5EiFEU/rxxx+ZNWsW/fv3Jy4ujuuvv54DBw7Uud1LL73EgAEDUEpx9uzZS7Zds2YNGzdurHdsqampPPTQQ/XezgqBlxTyjeUtIttLUhAiUGituemmm5g0aRKHDx8mLS2N3/72t5w+fbrObRMTE1m9ejV9+vSps+2lkoLT6ax1u4SEBF544YU6998UAu5qq5kUOoTbHIkQoql8++23hIaGcu+995qvxcfHe7XtqFHeVQk+duwYr732GsHBwbzzzju8+OKLvPHGG3To0IEdO3YwevRobrnlFhYvXkxRURERERGsXLmSQYMGsWbNGpYvX84//vEPnn76adLT0zly5Ajp6eksXry4SUcRgZcUCiQpCGGrxYth586621W08ea6Qnw8PP98rW/v2bOHyy+//KLX8/LymDhxYo3bvPfee8TFxdX9vd369u3LvffeS+vWrfnFL34BwBtvvMGBAwdYvXo1wcHB5ObmkpKSQkhICKtXr+bRRx/lo48+umhf+/fv59tvvyUvL49BgwZx3333NXjeQX0FXFIoyDf+lVtShRDR0dHs9CZBNcLMmTMJDjbmReXk5DBv3jwOHjyIUoqyspqLfU2ZMoWwsDDCwsLo0qULp0+fJiYmxtI4KwRcz1hYBEGUExYmk9eEsMUlPtF7qBghrFnT6G85dOhQPvzww4te9+VIoTZRUVHm4yeeeIKf/OQnrFq1imPHjjGpllFQWFjlNc/g4OBLXo/wtQBMCkFEqiKUam13KEKIJnLllVfy6KOP8vrrr7NgwQIAtm7dSmFhoU9HCtHR0eTm5tb6fk5ODj179gTgrbfe8tn39aXAu/uoOIjIIFl/RYhAopRi1apVfPPNN/Tv35+hQ4fy9NNP06NHjzq3feGFF4iJiSEjI4MRI0Ywf/78WtvecMMNrFq1ivj4eNatW3fR+7/85S955JFHSExMpLy8vFHHZBXVhFUwfSIhIUGnpqY2ePt5vf9FyulBHC3p6cOohBCXsm/fPoYMGVK/jXx4+ijQ1PTzVkpt01on1LVtwJ0+KigJJTK4xO4whBB1kWRgi4BLCoVlIUSFltodhhDCj61cuZI//vGPHq8lJiby8ssv2xSR7wRgUgglMqzm28CEEMIbd955J3feeafdYVgi8C40l7UislXT3d4lhBD+JOCSQkF5GJGtmudVfyGEsFvAJYVCVzhR4ZIUhBCiJgGZFCIjXHaHIYSog5RTsEdgJQWXi0IiiYywOxAhRFNrzvUUwFhl9b333mvQtr4UUElBFxYZSSHSvybsCSEapznUU6iLJAUblJwvxEUwkZFSK1aIQFJbPYXaFsOratSoUfTt27fOdhX1FJ577jlzmYusrCxmzJjBmDFjGDNmDBs2bABg7dq1xMfHEx8fz6hRo8jLy2PZsmWsW7eO+Ph4nnvuuQYfa2MF1DyFwnPGmkdRrSUpCGEXG8op2FZPYfbs2SxZsoSkpCTS09O55ppr2LdvH8uXL+fll18mMTGR/Px8wsPD+f3vf28W2rFTYCWF7CIAIqNl2WwhhPX1FFavXk1aWpr5PDc3l7y8PBITE3n44YeZM2cO06dPb7JaCd4IrKRw3ljzSJKCEPaxoZyCbfUUXC4XmzZtIiLC8+6WZcuWMWXKFL744gvGjRvH6tWrG/V9fCmgrikUXjDWPJKqa0IEliuvvJKSkhJef/1187WtW7eyfft2du7cWeNXQxJCdHQ0eXl55vOrr76al156yXxeMSo5fPgww4cP51e/+hUJCQns37//om3tElBJoeCCseZRZNumqXUqhGge7Kqn8MILL5CamsqIESOIi4vjtddeA+D5559n2LBhjBw5koiICK677jpGjBhBSEgII0eOtPVCc0DVU/jmqfVc/WwS6986ROK8AT6OTAhRm4bUU5ByCg0n9RS8VJhrLIQX2a6VzZEIIeoiycAegZUU8ow1jyLbh9XRUgghaif1FFqIwnxjzaPIDuE2RyKE8GdST6GFKHAnhajOkTZHIoQQzVNAJYXCAuNfuftICCFqFlhJoQgULsLkkoIQzd6kSZOYJGtnNzlLk4JS6lql1A9KqUNKqWW1tLlZKZWmlNqrlLJ0icDCIkWkKkLJ0kdCCFEjy5KCUioYeBm4DogDblVKxVVrMxB4BEjUWg8FFlsVD0BhcRCRqtjKbyGEaKZ8XU9Ba81DDz3EgAEDGDFiBNu3b691H41ZFnvChAkN2q6hrBwpjAUOaa2PaK1LgfeBadXaLABe1lqfB9Ban7EwHgqKg4kKLrLyWwghmiEr6il8+eWXHDx4kIMHD7JixQruu+++WvdxqaTgdDov+f0bWp+hoay8JbUncKLK8wzgimptLgNQSm0AgoGntdZfVd+RUmohsBCgd+/eDQ6osCSEyJDSBm8vhGi8xYsXe7UyaUUbb64rxMfH8/wlVtqrrZ6CN0aNGlXj659++im33347SinGjRvHhQsXOHXqFN27d7+o7bJly9i3bx/x8fHMmzeP9u3b8/nnn1NcXExBQQGfffYZ06ZN4/z585SVlfHrX/+aadOMz9CtW7cmPz+fNWvW8PTTT9OpUydzKfB33nkH5ePz4VYmhZoirb6mRggwEJgExADrlFLDtNYXPDbSegWwAoxlLhoaUGGpJAUhApEV9RQyMzPp1auX+TwmJobMzMwak0L1WglvvfUWmzZt4vvvv6dDhw44nU5WrVpFmzZtOHv2LOPGjWPq1KkXdfg7duxg79699OjRg8TERDZs2EBSUpJXPwNvWZkUMoBeVZ7HACdraLNZa10GHFVK/YCRJLZaEVBhWSiRrcqs2LUQwkuX+kRfVcUIYY2F6100pp5CTevG1edT++TJk+nQoYO5r0cffZSUlBSCgoLIzMzk9OnTdOvWzWObsWPHmrUX4uPjOXbsmF8lha3AQKVULJAJzAJmV2vzCXAr8JZSqhPG6aQjVgVU6GxFx9aSFIQINFbUU4iJieHEicoz5BkZGV6tulohKirKfPzuu++SlZXFtm3bCA0NpW/fvhQXX3xTTFiV++mDg4PrvB7REJZdaNZaO4EHgK+BfcAHWuu9SqlnlVJT3c2+BrKVUmnAt8BSrXW2VTEVlIcRGVZu1e6FEM2UFfUUpk6dyttvv43Wms2bN9O2bdsaTx3BxXUWqsvJyaFLly6Ehoby7bffcvz48YYdqA9YOk9Ba/2F1voyrXV/rfVv3K89qbX+zP1Ya60f1lrHaa2Ha63ftzKewvJwIsNcVn4LIUQzZEU9heuvv55+/foxYMAAFixYwCuvvFLrPuqqlTBnzhxSU1NJSEjg3XffZfDgwQ0/2EYKqHoK3YJOc+PwI7y2a7yPoxJCXEpD6imIhmtMPYXAWeZCawp1BJER/pUEhRCiKQXM0tm6uIQCoqhybUcIIXxq9+7dzJ071+O1sLAwvvvuO5siqr+ASQql5wtw0ZFIWTVbCGGR4cOHN/gW1+YiYE4fFWYby1tEtg6YQxZCiHoLmB6y8HwJIElBCCEuJWB6SDMpRAfbHIkQwiurJxlfokkFTFIoOG+seRTVRpKCEELUJmCSQuEFIylIKU4hApO/1lMA+O1vf9vgbesrcJJCrrFGSGS7VjZHIoRoas25noI3mjIpBMwtqZIUhGgmti2G817ctlnRxpvrCu3j4XL/qafw0EMPsWzZMtasWUNJSQn3338/99xzD6dOneKWW24hNzcXp9PJq6++yueff05RURHx8fEMHTqUd99916u4GyrgkkJUh7A6WgohWprmVk9hxYoVtG3blq1bt1JSUkJiYiJXX301H3/8Mddccw2PPfYY5eXlFBYWMnHiRF566aUmm/8QMEnh05XG4quRHcJtjkSIAHeJT/QeKkYIV62xKhLb6in885//5PvvvzeX887JyeHgwYOMGTOGu+66i7KyMm688UavRzO+FDBJoRzjriNJCkIEnuZWT0FrzYsvvsg111xz0XspKSl8/vnnzJ07l6VLl3L77bd7tU9fCZgLzam9pgMQ2V5OHwkRaJpbPYVrrrmGV199lbIyo+jXgQMHKCgo4Pjx43Tp0oUFCxZw9913m3c0hYaGmm2tFjBJISYGxo+H8AjfFrkWQjR/za2ewvz584mLi2P06NEMGzaMe+65B6fTyZo1a4iPj2fUqFF89NFHLFq0CICFCxcyYsQI5syZ45sfyCUETD0Fd7lXLCz3KoSohdRTaFqNqacQMNcUJBkIIUTdAiYpCCGE1aSeghBCCJPUUxBCCC/52/VLf9XYn7MkBSGE5cLDw8nOzpbEYDGtNdnZ2YSHN3w+lpw+EkJYruKWzqysLLtDafHCw8OJiYlp8PaSFIQQlgsNDSU2NtbuMIQX5PSREEIIkyQFIYQQJkkKQgghTH63zIVSKgs43sDNOwFn62zVssgxBwY55sDQmGPuo7XuXFcjv0sKjaGUSvVm7Y+WRI45MMgxB4amOGY5fSSEEMIkSUEIIYQp0JLCCrsDsIEcc2CQYw4Mlh9zQF1TEEIIcWmBNlIQQghxCX6bFJRS1yqlflBKHVJKLavh/TCl1F/d73+nlOpb5b1H3K//oJS6xtt92s3Xx6yU6qWU+lYptU8ptVcptajpjsY7Vvye3e8FK6V2KKX+Yf1R1I9Ff9vtlFIfKqX2u3/f45vmaLxj0TEvcf9d71FK/a9SquGrxFmgoceslOro/n+br5R6qdo2lyuldru3eUEpVf/6w1prv/sCgoHDQD+gFbALiKvW5v8Br7kfzwL+6n4c524fBsS69xPszT5b4DF3B0a720QDB1r6MVfZ7mHgPeAfdh9nUxwz8BdgvvtxK6Cd3cdq8d92T+AoEOFu9wFwh93H6qNjjgKSgHuBl6ptswUYDyjgS+C6+sbmryOFscAhrfURrXUp8D4wrVqbaRj/EQA+BH7qzprTgPe11iVa66PAIff+vNmnnXx+zFrrU1rr7QBa6zxgH8Z/pubCit8zSqkYYArw5yY4hvry+TErpdoADuANAK11qdb6QhMci7cs+T1jLPgZoZQKASKBkxYfR300+Ji11gVa6/VAcdXGSqnuQBut9SZtZIi3gRvrG5i/JoWewIkqzzO4uDMz22itnUAO0PES23qzTztZccwm99B0FNCc6gZadczPA78EXL4PudGsOOZ+QBaw0n3K7M9KqShrwm8Qnx+z1joTWA6kA6eAHK31Py2JvmEac8yX2mdGHfusk78mhZrOk1W/jaq2NvV9vbmw4piNjZRqDXwELNZa5zY4Qt/z+TErpX4GnNFab2tscBax4vccAowGXtVajwIKgOZ0zcyK33N7jE/asUAPIEopdVujovStxhxzY/ZZJ39NChlAryrPY7h4aGi2cQ8f2wLnLrGtN/u0kxXHjFIqFCMhvKu1/tiSyBvOimNOBKYqpY5hDNmvVEq9Y0XwDWTV33aG1rpiFPghRpJoLqw45quAo1rrLK11GfAxMMGS6BumMcd8qX1Wra7TsD7M7gsuDbxIEwIcwfgUUHGRZmi1NvfjeZHmA/fjoXhemDqCcdGnzn22wGNWGOcdn7f7+JrqmKttO4nmd6HZkmMG1gGD3I+fBv5g97Fa/Ld9BbAX41qCwjg3/6Ddx+qLY67y/h1cfKF5KzCOygvN19c7Nrt/OI34oV6PcbfMYeAx92vPAlPdj8OBv2FceNoC9Kuy7WPu7X6gytX5mvbZnL58fcwYdzBo4Htgp/ur3n9E/nTM1fY9iWaWFCz8244HUt2/60+A9nYfZxMc8zPAfmAP8D9AmN3H6cNjPoYxasjHGCHEuV9PcB/vYeAl3BOU6/MlM5qFEEKY/PWaghBCCAtIUhBCCGGSpCCEEMIkSUEIIYRJkoIQQgiTJAUhhBAmSQpCCCFMIXYHIERzoJR6ApiDsQDZWWAbxgJkCzFmnB4C5mqtC5VSbwFFwGCgD3AnMA9jyeLvtNZ3uPeZD7yMseTCeeBR4D+B3hjrTH3mXojwfzCWQwZ4QGu90dqjFaJ2MlIQAU8plQDMwFgldjrGrFCAj7XWY7TWIzGWFb+7ymbtgSuBJcDfgecwllwYrpSKd7eJAtZorS8H8oBfA5OBmzBmrgKcASZrrUcDtwAvWHKQQnhJRgpCGMt9fKq1LgJQSv3d/fowpdSvgXZAa+DrKtv8XWutlVK7gdNa693ubfcCfTGWDCkFvnK33w2UaK3L3Nv0db8eCrzkTiTlwGXWHKIQ3pGkIETNSw4DvAXcqLXepZS6A2OtpAol7n9dVR5XPK/4f1WmK9eRMdtprV3uVS/BGGmcBkZijNw9CqcI0dTk9JEQsB64QSkV7q4tMcX9ejRwyr28+ByLvndb4JTW2gXMxVjhUwjbyEhBBDyt9Val1GcYyxcfx1hNNAd4AqMS3XGM0z/RFnz7V4CPlFIzgW8xCuAIYRtZJVUIjOpzWut8pVQkkAIs1O761UIEEhkpCGFYoZSKw1jD/i+SEESgkpGCEEIIk1xoFkIIYZKkIIQQwiRJQQghhEmSghBCCJMkBSGEECZJCkIIIUz/H5EzHPcKFrQ9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.errorbar(x=param2[\"gamma\"],y=clf2.cv_results_['mean_train_score'][:4],yerr=clf2.cv_results_['std_train_score'][:4],label='C=1_train',color=\"red\")\n",
    "plt.errorbar(x=param2[\"gamma\"],y=clf2.cv_results_['mean_test_score'][:4],yerr=clf2.cv_results_['std_test_score'][:4],label='C=1_test',color=\"blue\")\n",
    "plt.errorbar(x=param2[\"gamma\"],y=clf2.cv_results_['mean_train_score'][4:],yerr=clf2.cv_results_['std_train_score'][4:],label='C=10_train',color=\"black\")\n",
    "plt.errorbar(x=param2[\"gamma\"],y=clf2.cv_results_['mean_test_score'][4:],yerr=clf2.cv_results_['std_test_score'][4:],label='C=10_test',color=\"orange\")\n",
    "plt.legend()\n",
    "plt.xlabel(\"gamma\")\n",
    "plt.ylabel(\"score\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于固定的C，gamma值越大对应高斯核的方差越小，模型越复杂，从而对训练集的得分越高"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同C的情况类似，当随着gamma继续增大时，模型变得过于复杂，从而出现了过拟合现象，效果下降"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
